Model API
A provider owns the model server and GPUs. Your product sends requests.
OperationsLow
- Choose it when
- You need to ship, test demand, or support changing usage.
- Watch for
- Usage spend, data policy, and provider limits.
Model hosting, explained
Where does an AI model actually run?
Your app does not need a GPU just because it uses AI. First decide whether another company serves the model, a GPU wakes up on demand, or you keep the hardware running yourself.
Your serverApp or agentCPU is usually enough
API call→
Inference serverOpen modelGPU is used here▦
Rule of thumbSeparate orchestration from inference before choosing hardware.
Common assumption“I am building an agent, so I need a GPU.”
→
Better mental modelAgent = CPU workflow
Model = API or GPU
Model = API or GPU
Choose the operating model
Same AI. Very different job.
Move right only when the extra control solves a real requirement. Complexity has a monthly cost even when the dashboard calls the GPU cheap.
Serverless GPU
Your model image runs on rented GPU workers that can scale down.
OperationsMedium
- Choose it when
- You need custom weights, burst jobs, or an isolated endpoint.
- Watch for
- Cold starts, storage, worker minimums, and quotas.
Always-on GPU
You rent a GPU instance and operate the full serving stack.
OperationsHigh
- Choose it when
- Load is steady and measured, or dedicated capacity is required.
- Watch for
- Idle bill, availability, drivers, security, and scaling.
Fast capacity check
Model size → first GPU to test
For 4-bit inference, short context, and one active sequence. This answers “might it fit?”, not “will it be fast enough?”
Estimated8Bmodel class
16 GBVRAM target
Estimated24Bmodel class 24 GBVRAM target
Estimated32Bmodel class 32 GBVRAM target
Estimated70Bmodel class 48 GBVRAM target
Sourced + estimated
Parameter counts are sourced from official model cards. Weight memory and first-test tiers are transparent estimates. Quantization metadata, runtime, KV cache, long context, batching, and concurrent users need headroom.
Open the complete VRAM guide →Before renting hardware
Five questions. In this order.
If you cannot answer traffic and control requirements yet, an API is a reversible default.
- 1Do I need custom weights?No → begin with an API.
- 2Must inference stay in my environment?No → keep outsourcing GPU operations.
- 3Is traffic bursty or steady?Bursty → serverless. Steady → price a warm instance.
- 4What actually fits in VRAM?Include context, cache, and concurrency — not only weights.
- 5What did the exact workload measure?Use peak VRAM, latency, utilization, and total monthly bill.
Deep research
Model hosting research library
All the detailed provider comparisons, pricing source trails, implementation caveats, and experiment notes remain available here without dominating the page.
ai hosting A100 rental prices 2026: verified rates and a RunPod conflict Compare dated A100 40 GB and 80 GB rental rates, corrected Thunder CPU inclusions, higher Verda prices, and an unresolved RunPod Secure Cloud price conflict. 2026-10-15 · Read the evidence → ai hosting Serverless GPU pricing 2026: rates, idle costs and bill checks Compare H100 rates and idle-cost rules, with Northflank GPU prices checked October 2026 and a service-level billing walkthrough that avoids account-total traps. 2026-10-15 · Read the evidence → ai hosting Baseten pricing 2026: GPU rates and deployment bill checks Baseten GPU rates verified October 2026, warm-replica cost estimates, and a billing API walkthrough separating compute, surcharges, and credits. 2026-10-13 · Read the evidence → ai hosting H100 rental price per hour in 2026: 19 rates and variant costs Compare 19 H100 rates and Massed Compute’s $2.73 base versus $3.14 SXM5 VM, a $295.20 difference over 720 hours. 2026-10-12 · Read the evidence → ai hosting Modal pricing 2026: GPU rates, concurrency ceilings, and idle-tail costs Modal GPU rates rechecked October 10, 2026: 11 GPU prices, Starter and Team concurrency limits, and the CPU, memory, region, and idle costs beyond the GPU rate. 2026-10-10 · Read the evidence → ai hosting GPU cloud cost calculator 2026: A100 rates, stop costs and retained storage Eight dated A100 rates, including Verda at $1.870/hour on October 9, with stop and retained-storage costs for an eight-hour job. 2026-10-09 · Read the evidence → ai hosting What GPU do you need to run Llama 70B? VRAM, context, and KV-cache guide Llama 70B GPU VRAM and KV-cache math, selected cloud costs, and why a listed Koyeb B200 price is not deployable-capacity evidence. 2026-10-06 · Read the evidence → ai hosting RunPod pricing 2026: 42 Pod prices, 13 Serverless tiers, and stopped-Pod costs RunPod pricing checked October 2026: 42 Pod prices, 13 Serverless tiers, stopped-Pod storage costs, restart risk, and break-even math. 2026-10-02 · Read the evidence → ai hosting GPU cloud free credits 2026: which trials stop before charging? A source-checked comparison of GPU cloud credits, GPU access gates, paid-account transitions, and the controls that can stop trial spend. 2026-09-27 · Read the evidence →
Show 25 more research notes
Replicate pricing 2026: GPU rates, idle-tail costs, and deployment floors 2026-09-20 Replicate for AI inference APIs and jobs (July 2026): fast to ship, expensive once you buy warm control 2026-09-03 Fal vs Baseten vs Modal for one warm H100 deployment (July 2026): cheapest managed floor or best serving surface? 2026-07-21 Fal for AI inference APIs and jobs (July 2026): serious serverless deployment controls, premium warm-GPU economics 2026-07-20 Modal vs RunPod for one warm inference endpoint (July 2026): cleaner Python ergonomics or cheaper GPU control? 2026-07-19 Baseten vs RunPod for one warm inference endpoint (July 2026): managed inference polish or cheaper GPU control? 2026-07-18 Baseten vs Modal vs Replicate for one warm inference endpoint (July 2026): who makes you pay for readiness? 2026-07-17 Baseten for AI inference APIs and jobs (July 2026): polished dedicated inference, pricey once replicas stay warm 2026-07-16 DigitalOcean vs Hetzner Cloud for AI side projects (July 2026): clean cloud vs cheap cloud 2026-07-11 RunPod for AI inference APIs and jobs (July 2026): flexible GPU hosting with sharp billing edges 2026-07-11 Best hosting for AI agents on a budget (June 2026): choose by workload, not by AI branding 2026-07-10 DigitalOcean Droplets for AI side projects (June 2026): what fits, what breaks, and when to pay for dedicated CPU 2026-07-10 DigitalOcean vs Hetzner vs Hostinger for AI side projects (June 2026): the honest always-on floor 2026-07-10 Hetzner Cloud for AI side projects (July 2026): cheapest serious self-hosted CPU, with the catches that still matter 2026-07-10 Modal for AI inference APIs and jobs (July 2026): brilliant for bursty GPU work, awkward as a cheap warm endpoint 2026-07-08 Hetzner vs Contabo vs Hostinger VPS for AI workloads (June 2026): which budget box actually fits an agent stack 2026-06-29 S3 alternatives for AI assets: R2 vs B2 vs Wasabi vs Tigris (June 2026) 2026-06-29 What it costs to run an AI side project on a VPS for 30 days (June 24, 2026): honest budget ranges 2026-06-24 RunPod vs Modal vs Replicate for shipping a small inference API (June 2026): who should own the endpoint, queue, and warm pool? 2026-06-22 Cloudflare Workers AI vs self-hosted GPU: when each wins (June 2026) 2026-06-18 Best VPS setup for LangGraph or CrewAI (June 2026): what fits on 4 GB, 8 GB, and beyond 2026-06-13 Hostinger VPS for AI side projects: what fits, what breaks, and when to upgrade (June 2026) 2026-06-03 Best hosts for long-running agent workers (June 2026): where always-on costs and queue limits bite 2026-06-02 Vector database hosting for small AI apps (May 2026): when pgvector wins, when Qdrant wins, and when managed vector DB is worth it 2026-05-25 OpenRouter vs Together vs Groq vs Fireworks vs Cerebras: the per-token model gateways compared (April 2026) 2026-04-27