šÆ What You Will Learn
Without a health endpoint, your orchestrator can't tell a hung app from a healthy one. Without metrics, you find out about an outage from users. We'll fix both: proper liveness/readiness checks and Prometheus metrics that make your API observable.
- Why "no health check" means killed pods and silent outages
- Liveness vs readiness ā the difference that trips everyone up
- A fast
/healthand a dependency-checking/health/ready - Exposing Prometheus
/metrics(the RED method) - Adding your own custom business metrics
- Wiring Kubernetes probes and a Prometheus scrape
Prerequisites: a FastAPI app with a database (how-to-connect-fastapi-to-postgres). This is the metrics half of observability ā pair it with structured logs from fastapi-logging.
fastapi-health-metrics/
āāā requirements.txt
āāā app/
āāā __init__.py
āāā database.py # get_session
āāā metrics.py # custom Prometheus metrics
āāā main.py # /health, /health/ready, /metricsThe Problem: Flying Blind (See It First)
Ship a FastAPI app with no health endpoint and no metrics, and you've created two blind spots at once:
No health endpoint:
Kubernetes/Docker: "is this container alive? ...I'll just assume yes."
ā a hung app keeps receiving traffic; a slow-starting app gets traffic
before it's ready ā users hit errors
No metrics:
"Is the API slow right now? How many errors in the last 5 min?"
ā you have no idea until a user complains. You're debugging from logs
alone, after the fact.Liveness vs Readiness (Don't Mix Them Up)
These are two different questions, and conflating them causes real outages. Liveness asks "is the process alive?" Readiness asks "can it serve traffic right now?"
| Liveness | Readiness | |
|---|---|---|
| Question | Is the process alive? | Can it serve requests now? |
| Checks dependencies (DB)? | ā No | ā Yes |
| Fails ā orchestrator... | restarts the pod | stops routing traffic (no restart) |
| Should be | Dumb & instant | Checks critical deps |
| Endpoint | /health | /health/ready |
Liveness: A Dumb, Fast /health
Liveness should do nothing but confirm the app can respond. No DB, no external calls ā just proof the event loop is running. Keep it trivial so it's fast and never falsely fails.
from fastapi import FastAPI
app = FastAPI(title="FastAPI Health & Metrics", version="0.1.0")
@app.get("/health", tags=["health"])
def liveness():
# if this returns, the process is alive. That's the whole job.
return {"status": "ok"}Readiness: Check Critical Dependencies
Readiness is where you verify the app can actually do its job ā most importantly, that it can reach the database. If a dependency is down, return 503 so the orchestrator stops routing traffic until it recovers.
from typing import Annotated
from fastapi import Depends, HTTPException, status
from sqlalchemy import text
from sqlmodel import Session
from app.database import get_session
SessionDep = Annotated[Session, Depends(get_session)]
@app.get("/health/ready", tags=["health"])
def readiness(session: SessionDep):
try:
session.execute(text("SELECT 1")) # can we reach the DB?
except Exception:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="database unavailable",
)
return {"status": "ready"}Metrics 101: Measure the Right Things (RED)
Before adding metrics, know what matters. The RED method is the standard starting point for a request-driven service ā three signals that tell you almost everything about its health:
| Signal | Question it answers | Prometheus type |
|---|---|---|
| Rate | How many requests per second? | Counter |
| Errors | How many are failing (5xx)? | Counter (by status) |
| Duration | How long do they take (p95/p99)? | Histogram |
/metrics endpoint in a text format, and a Prometheus server scrapes it on an interval. You don't push anywhere ā you just expose the current numbers and let Prometheus collect them.Exposing Prometheus Metrics
The prometheus-fastapi-instrumentator library gives you all the RED metrics ā request count, status codes, and a latency histogram ā and a /metrics endpoint in two lines. It hooks into the request cycle for you.
from prometheus_fastapi_instrumentator import Instrumentator
# after all routes are defined:
Instrumentator().instrument(app).expose(app) # adds GET /metricsHit /metrics and you get the standard Prometheus exposition format:
$ curl -s http://127.0.0.1:8000/metrics | grep http_request
# HELP http_requests_total Total number of requests by method, status and handler.
http_requests_total{handler="/health",method="GET",status="2xx"} 42.0
http_requests_total{handler="/users",method="POST",status="2xx"} 7.0
# HELP http_request_duration_seconds Duration of HTTP requests in seconds.
http_request_duration_seconds_bucket{handler="/users",le="0.1"} 6.0
http_request_duration_seconds_bucket{handler="/users",le="0.5"} 7.0/metrics and you can graph request rate, error rate, and p95 latency for the whole API.Custom Business Metrics
HTTP metrics tell you the API is healthy; business metrics tell you the product is. Add your own with prometheus-client (already installed as a dependency). A Counter for signups, a Gauge for something you can go up and down ā they show up on the same /metrics.
from prometheus_client import Counter, Gauge
# name convention: app_<thing>_total for counters
signups_total = Counter(
"app_signups_total",
"Total number of successful signups",
)
active_websocket_connections = Gauge(
"app_active_websocket_connections",
"Currently open WebSocket connections",
)from app.metrics import signups_total
@app.post("/signup", status_code=status.HTTP_201_CREATED)
def signup(session: SessionDep):
# ...create the user...
signups_total.inc() # count a real business event
return {"status": "created"}$ curl -s http://127.0.0.1:8000/metrics | grep app_signups # HELP app_signups_total Total number of successful signups # TYPE app_signups_total counter app_signups_total 3.0
Try It
fastapi==0.116.1
uvicorn[standard]==0.30.6
sqlmodel==0.0.22
psycopg[binary]==3.2.1
prometheus-fastapi-instrumentator==7.0.0python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload# liveness ā always fast, no deps
$ curl -s http://127.0.0.1:8000/health
{"status":"ok"}
# readiness ā checks the DB
$ curl -s http://127.0.0.1:8000/health/ready
{"status":"ready"}
# stop Postgres, then readiness fails (liveness stays OK!)
$ curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/health/ready
503
# metrics
$ curl -s http://127.0.0.1:8000/metrics | head -n 5/metrics for Prometheus to scrape.Common Mistakes (and Fixes)
| Mistake | Symptom | Fix |
|---|---|---|
| Checking the DB in liveness | DB blip restarts every pod (crash loop) | Deps go in readiness only |
| One /health for both probes | Can't distinguish alive vs ready | Separate /health and /health/ready |
| Readiness pings every external API | One slow 3rd-party pulls you offline | Check only critical deps, with a timeout |
| Rate-limiting the health endpoints | Probes get 429, app marked unhealthy | Exempt them (fastapi-rate-limiting) |
/metrics public on the internet | Leaks internal traffic patterns | Restrict to internal network / scraper |
| Metric labels with unbounded values | Cardinality explosion, Prometheus OOM | Never label by user id, path params, etc. |
Production Notes
Wire the probes in Kubernetes so each does its job:
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /health/ready
port: 8000
initialDelaySeconds: 5
periodSeconds: 5And point Prometheus at the app:
scrape_configs:
- job_name: fastapi
metrics_path: /metrics
static_configs:
- targets: ["fastapi:8000"]- Beware metric cardinality. Never put high-variety values (user id, order id, raw path) in labels ā each combination is a new time series and will melt Prometheus. The instrumentator groups by route template (
/users/{id}), not the actual id, on purpose. - Protect
/metrics. Keep it on an internal network or behind auth ā it reveals traffic and error patterns you don't want public. - Metrics + logs + traces. Metrics tell you something is wrong; structured logs (
fastapi-logging) tell you what. Use them together.
What's Next
fastapi-loggingā the structured-logs half of observabilityfastapi-dockerā where the health checks get wired into the containerfastapi-github-actionsā CI that ships the observable imagefastapi-deploymentā run it behind an orchestrator that uses these probes
Recap: give your app a dumb-and-fast /health for liveness and a dependency-checking /health/ready for readiness ā never check the DB in liveness. Expose Prometheus /metrics for the RED signals, add custom counters for business events, and watch your cardinality. Now your orchestrator can heal the app, and you can actually see how it's doing.
fastapi-health-metrics/ # ā
finished
āāā requirements.txt # + prometheus-fastapi-instrumentator
āāā app/
āāā database.py # get_session
āāā metrics.py # custom Counter/Gauge
āāā main.py # /health (liveness), /health/ready, /metrics