Model API
A provider owns the model server and GPUs. Your product sends requests.
OperationsLow
- Choose it when
- You need to ship, test demand, or support changing usage.
- Watch for
- Usage spend, data policy, and provider limits.
Model hosting, explained
Where does an AI model actually run?
Your app does not need a GPU just because it uses AI. First decide whether another company serves the model, a GPU wakes up on demand, or you keep the hardware running yourself.
Your serverApp or agentCPU is usually enough
API call→
Inference serverOpen modelGPU is used here▦
Rule of thumbSeparate orchestration from inference before choosing hardware.
Common assumption“I am building an agent, so I need a GPU.”
→
Better mental modelAgent = CPU workflow
Model = API or GPU
Model = API or GPU
Choose the operating model
Same AI. Very different job.
Move right only when the extra control solves a real requirement. Complexity has a monthly cost even when the dashboard calls the GPU cheap.
Serverless GPU
Your model image runs on rented GPU workers that can scale down.
OperationsMedium
- Choose it when
- You need custom weights, burst jobs, or an isolated endpoint.
- Watch for
- Cold starts, storage, worker minimums, and quotas.
Always-on GPU
You rent a GPU instance and operate the full serving stack.
OperationsHigh
- Choose it when
- Load is steady and measured, or dedicated capacity is required.
- Watch for
- Idle bill, availability, drivers, security, and scaling.
Fast capacity check
Model size → first GPU to test
For 4-bit inference, short context, and one active sequence. This answers “might it fit?”, not “will it be fast enough?”
Estimated8Bmodel class
16 GBVRAM target
Estimated24Bmodel class 24 GBVRAM target
Estimated32Bmodel class 32 GBVRAM target
Estimated70Bmodel class 48 GBVRAM target
Sourced + estimated
Parameter counts are sourced from official model cards. Weight memory and first-test tiers are transparent estimates. Quantization metadata, runtime, KV cache, long context, batching, and concurrent users need headroom.
Open the complete VRAM guide →Before renting hardware
Five questions. In this order.
If you cannot answer traffic and control requirements yet, an API is a reversible default.
- 1Do I need custom weights?No → begin with an API.
- 2Must inference stay in my environment?No → keep outsourcing GPU operations.
- 3Is traffic bursty or steady?Bursty → serverless. Steady → price a warm instance.
- 4What actually fits in VRAM?Include context, cache, and concurrency — not only weights.
- 5What did the exact workload measure?Use peak VRAM, latency, utilization, and total monthly bill.
Deep research
Model hosting research library
All the detailed provider comparisons, pricing source trails, implementation caveats, and experiment notes remain available here without dominating the page.
ai hosting GPU cloud free credits 2026: what $30-$300 can actually test An August 2026 source check of GPU cloud credits, account restrictions, and the real GPU time Modal's recurring $30 can fund. 2026-08-16 · Read the evidence → ai hosting A100 rental price per hour in 2026: 12 public rates checked A100 rental prices across 12 public rates, with separate 40 GB and 80 GB comparisons plus the billing catches behind the cheapest offers. 2026-08-15 · Read the evidence → ai hosting H100 rental price per hour in 2026: 10 public cloud rates checked H100 price per hour across 10 public rates, with GPU variants, single-GPU eligibility, stopped-VM billing, and 720-hour cost estimates. 2026-08-13 · Read the evidence → ai hosting Modal pricing 2026: GPU rates, plan fees, and warm-container costs Modal pricing explained with August 2026 GPU rates, Starter and Team limits, supporting-resource charges, and warm-container cost estimates. 2026-08-12 · Read the evidence → ai hosting RunPod pricing 2026: Pods vs Serverless break-even costs RunPod pricing explained with August 2026 Pod and Serverless rates, break-even worker hours, storage charges, and prepaid-balance risks. 2026-08-12 · Read the evidence → ai hosting GPU cloud cost calculator (2026): warm endpoint vs 1%-utilization job A source-backed GPU cloud cost calculator refresh: use the same public H100 rate to model an always-warm endpoint, a 10% duty-cycle worker, and a genuinely bursty 1% job. 2026-08-11 · Read the evidence → ai hosting Serverless GPU pricing in 2026: August 10 rates and deployment matrix An August 10, 2026 source-backed GPU deployment price matrix that separates Pods, serverless workers, managed inference, and GPU VMs instead of treating unlike rates as one market. 2026-08-10 · Read the evidence → ai hosting What GPU do you need to run Llama 70B? VRAM guide for self-hosting open models A source-backed Llama 70B VRAM guide covering 4-bit, 8-bit, and BF16 memory estimates, quantization caveats, and practical current cloud GPU capacity options. 2026-07-29 · Read the evidence → ai hosting Baseten pricing 2026: warm-replica costs explained Baseten pricing explained: live per-minute GPU rates, 30-day warm-replica estimates, and when scale-to-zero really lowers the bill. 2026-07-26 · Read the evidence →
Show 25 more research notes
Replicate pricing 2026: public models vs deployments 2026-07-26 Fal vs Baseten vs Modal for one warm H100 deployment (July 2026): cheapest managed floor or best serving surface? 2026-07-21 Fal for AI inference APIs and jobs (July 2026): serious serverless deployment controls, premium warm-GPU economics 2026-07-20 Modal vs RunPod for one warm inference endpoint (July 2026): cleaner Python ergonomics or cheaper GPU control? 2026-07-19 Baseten vs RunPod for one warm inference endpoint (July 2026): managed inference polish or cheaper GPU control? 2026-07-18 Baseten vs Modal vs Replicate for one warm inference endpoint (July 2026): who makes you pay for readiness? 2026-07-17 Baseten for AI inference APIs and jobs (July 2026): polished dedicated inference, pricey once replicas stay warm 2026-07-16 Replicate for AI inference APIs and jobs (July 2026): fast to ship, expensive once you buy warm control 2026-07-12 DigitalOcean vs Hetzner Cloud for AI side projects (July 2026): clean cloud vs cheap cloud 2026-07-11 RunPod for AI inference APIs and jobs (July 2026): flexible GPU hosting with sharp billing edges 2026-07-11 Best hosting for AI agents on a budget (June 2026): choose by workload, not by AI branding 2026-07-10 DigitalOcean Droplets for AI side projects (June 2026): what fits, what breaks, and when to pay for dedicated CPU 2026-07-10 DigitalOcean vs Hetzner vs Hostinger for AI side projects (June 2026): the honest always-on floor 2026-07-10 Hetzner Cloud for AI side projects (July 2026): cheapest serious self-hosted CPU, with the catches that still matter 2026-07-10 Modal for AI inference APIs and jobs (July 2026): brilliant for bursty GPU work, awkward as a cheap warm endpoint 2026-07-08 Hetzner vs Contabo vs Hostinger VPS for AI workloads (June 2026): which budget box actually fits an agent stack 2026-06-29 S3 alternatives for AI assets: R2 vs B2 vs Wasabi vs Tigris (June 2026) 2026-06-29 What it costs to run an AI side project on a VPS for 30 days (June 24, 2026): honest budget ranges 2026-06-24 RunPod vs Modal vs Replicate for shipping a small inference API (June 2026): who should own the endpoint, queue, and warm pool? 2026-06-22 Cloudflare Workers AI vs self-hosted GPU: when each wins (June 2026) 2026-06-18 Best VPS setup for LangGraph or CrewAI (June 2026): what fits on 4 GB, 8 GB, and beyond 2026-06-13 Hostinger VPS for AI side projects: what fits, what breaks, and when to upgrade (June 2026) 2026-06-03 Best hosts for long-running agent workers (June 2026): where always-on costs and queue limits bite 2026-06-02 Vector database hosting for small AI apps (May 2026): when pgvector wins, when Qdrant wins, and when managed vector DB is worth it 2026-05-25 OpenRouter vs Together vs Groq vs Fireworks vs Cerebras: the per-token model gateways compared (April 2026) 2026-04-27