Why FastAPI Apps Slow Down Over Time (Low CPU, High Latency Explained)

The “Silent Death” Scenario

You deploy your FastAPI application. For the first few hours, it feels blazing fast. Your P99 latency is sitting comfortably at 20ms. You high-five your team and go to sleep.

The next morning, your alerts are screaming.

• Latency: Spiking to 3,000ms+ randomly.

• Health Checks: Failing intermittently.

• The Weird Part: You check your dashboards, and CPU usage is at 5%. Memory is flat. The database is responding instantly.

If your server isn’t out of CPU and isn’t out of memory, why is it crawling?

The answer lies in The Event Loop. When CPU is low but latency is high, it usually means your application isn’t working, it is waiting.

This guide focuses on real-world scenarios where FastAPI apps slow down over time despite low CPU usage, a pattern that only appears under sustained load.

TL;DR: If your FastAPI app has low CPU but high latency, your event loop is blocked. The fix is not scaling CPUs, it’s fixing lifecycle, async misuse, and resource ownership.

Understanding the Single-Threaded Model

To fix the problem, you must first understand the machine.

FastAPI (running on Uvicorn) relies on Python’s asyncio event loop. Think of the Event Loop as a single waiter in a busy restaurant.

• Async: When a customer orders food (an API request), the waiter sends the order to the kitchen (Database/External API) and immediately goes to serve the next table. He doesn’t wait in the kitchen. This is why FastAPI can handle thousands of requests with one thread.

• Sync (Blocking): If the waiter decides to cook the food himself (Heavy Calculation) or stand in the kitchen staring at the chef (Synchronous I/O), he ignores all other tables.

If that waiter stops for just 1 second to chop an onion, every other customer waits 1 second longer. In a high-throughput system, these delays stack up exponentially.

**Mental Model**
Think of the event loop as a single-lane highway:
- Async = cars exit quickly, highway stays free
- Blocking = one broken car stops everything behind it

Killer #1: The “Sync” Blocker (and the def Trap)

This is the most common mistake developers make when migrating from Flask or Django.

In FastAPI, the moment you define an endpoint with async def, you are promising FastAPI: “I will not block the main thread.” FastAPI trusts you and runs that code directly on the main event loop.

If you put anything blocking inside an async def, you freeze the entire server.

The Mistake: Synchronous I/O in Async Path

Python

import time
import requests  # <--- The silent killer

@app.get("/external-data")
async def get_data():
    # BLOCKING!
    # This freezes the entire server. No other request can enter.
    response = requests.get("https://example.com/huge-data")
    time.sleep(1) # Simulating processing
    return response.json()

If 50 users hit this endpoint simultaneously, the 50th user doesn’t wait 1 second. They wait 50 seconds (1s x 50). The CPU usage remains low because the CPU isn’t computing - it’s just idle, waiting for the network response.

The Nuance: def vs async def

FastAPI has a safety valve. If you define your endpoint with just def (no async), FastAPI assumes it might block and runs it in a separate thread pool (Starlette’s run_in_threadpool).

Python

@app.get("/safe-sync")
def safe_sync():
    # Safe! FastAPI runs this in a thread pool.
    time.sleep(1)
    return {"status": "ok"}

However, you lose the benefits of async concurrency. The best solution is to keep async def but offload the blocking work.

The Fix: asyncio.to_thread

If you must use a blocking library (like requests or pandas) or perform a heavy calculation (like image processing), run it in a non-blocking way.

Python

import asyncio
import time

@app.get("/heavy-async")
async def heavy_async():
    # ✅ Runs in a separate thread, leaving the Event Loop free
    await asyncio.to_thread(time.sleep, 1)
    return {"status": "done"}

Red Flags That Your Event Loop Is Blocked

If you see any of these, suspect blocking immediately:

• Health checks timing out while business endpoints are slow

• Latency spikes without CPU spikes

• Restarting pods “fixes” performance temporarily

• Throughput collapses before memory or CPU exhaustion

Killer #2: The Hidden CPU Crushers (Password Hashing)

Sometimes, the blocker isn’t I/O; it’s pure CPU work.

A classic example is Password Hashing (bcrypt or argon2). Hashing a password is designed to be slow and CPU-intensive to thwart hackers.

The Mistake:

Python

from passlib.context import CryptContext

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

@app.post("/login")
async def login(user: UserLogin):
    # BLOCKING!
    # Verifying a hash takes ~300ms of pure CPU time.
    if not pwd_context.verify(user.password, user.hashed_password):
        raise HTTPException(...)
    return {"token": "..."}

If 10 users try to login at once, your event loop locks up for 3 seconds (300ms * 10). Your health check endpoint (which should take 1ms) will time out, causing Kubernetes to kill your pod.

Password hashing belongs to the request lifecycle, but not to the event loop.

The Fix: Even though this is CPU-bound, not I/O-bound, you must still move it off the main thread.

Python

@app.post("/login")
async def login(user: UserLogin):
    # Offload CPU work to a thread
    is_valid = await asyncio.to_thread(
        pwd_context.verify, user.password, user.hashed_password
    )
    if not is_valid:
        raise HTTPException(...)

Killer #3: The httpx Client Leak

If your API acts as a gateway or aggregator calling other services, you are likely using httpx or aiohttp. A common mistake is treating the async client like a standard variable.

HTTP clients should be treated like database engines: created once, owned by the application lifecycle, and cleaned up deterministically.

The Mistake: Creating a Client per Request

Python

@app.get("/proxy")
async def proxy_request():
    # Creates a new Connection Pool + SSL Context every time!
    async with httpx.AsyncClient() as client:
        resp = await client.get("https://example.com")
    return resp.json()

Why this kills performance:

1. TCP Handshake: Every request performs a 3-way handshake.

2. SSL Handshake: Expensive crypto negotiation happens every time.

3. File Descriptors: You will eventually run out of sockets (OSError: [Errno 24] Too many open files).

The Fix: The Lifespan Pattern Create one client at startup and reuse it. The connection pool stays open, making subsequent requests lightning fast.

Python

from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Initialize one client for the whole app
    app.state.http_client = httpx.AsyncClient()
    yield
    # Cleanup on shutdown
    await app.state.http_client.aclose()

app = FastAPI(lifespan=lifespan)

@app.get("/proxy")
async def proxy_request(request: Request):
    client = request.state.http_client
    resp = await client.get("https://example.com")
    return resp.json()

Killer #4: The “Zombie” Background Task

FastAPI provides BackgroundTasks, which are great. But often, developers want “fire and forget” logic using standard AsyncIO.

The Mistake:

Python

@app.post("/process")
async def process():
    # DANGER!
    # The Python Garbage Collector might destroy this task
    # immediately because no variable references it.
    asyncio.create_task(long_running_job())
    return {"status": "started"}

Not only can this task fail silently, but if it hangs, you have no way to cancel it or monitor it.

The Fix: The Set-Reference Pattern You must maintain a “Hard Reference” to your running tasks.

Python

# A global set to hold references
active_tasks = set()

async def safe_process():
    task = asyncio.create_task(long_running_job())

    # 1. Add to set (keep it alive)
    active_tasks.add(task)

    # 2. Remove from set when done (prevent memory leak)
    task.add_done_callback(active_tasks.discard)

Why Adding More CPU or Pods Won’t Save You

Auto-scaling hides event loop bugs, it doesn’t fix them.

When the loop is blocked:

• Each new pod reproduces the same problem

• Load balancers distribute latency, not relief

• Costs go up while performance stays bad

Until the event loop is unblocked, scaling is cosmetic.

How to Debug: Finding the Invisible Wall

You can’t see these issues with standard access logs because the logs only show total time. They don’t show waiting time.

You need a sampling profiler. The best tool for Python is py-spy. It reads memory directly, so it has almost zero overhead and works on production servers.

1. Install py-spy:

Bash

pip install py-spy

2. Peek at the running process: Find your uvicorn Process ID (PID) and run:

Bash

# --rate 100: samples 100 times per second
# --subprocesses: checks worker processes too
sudo py-spy top --pid <PID> --rate 100 --subprocesses

3. Interpret the Output:

• If you see time.sleep, select, or requests at the top of the list: You are blocking the loop.

• If you see _run_until_complete dominating without specific function calls: You might be starving the loop with too many tasks.

Conclusion

Low CPU + High Latency is almost always a symptom of a Blocked Event Loop.

FastAPI apps don’t decay randomly.
They slow down because something is blocking the event loop.

Once you understand resource ownership, what runs on the loop, what must not, and what belongs in the application lifespan performance issues stop being mysterious.

If you’re also seeing database pool exhaustion or connection timeouts, read my guide on Fixing QueuePool Overflow Errors in FastAPI, where lifecycle bugs surface again in a different form.

To keep your FastAPI app responsive:

1. Audit your async def endpoints: Ensure no requests, time.sleep, or heavy math exists there.

2. Use asyncio.to_thread: For anything blocking or CPU-intensive (like password hashing).

3. Reuse HTTP Clients: Treat httpx.AsyncClient like a database connection - create it once, use it everywhere.

If you are struggling with database connection pool limits (another common cause of latency), read my guide on Fixing QueuePool Overflow errors.