Our p99 latency was 2 seconds. Across all our APIs. Two full seconds before a user gets a response. And that was the average. Some endpoints clocked in at 17 seconds. A few hit 50. The only API that was anywhere near acceptable sat at 100ms, and honestly, 100ms is just barely not embarrassing.
The timeouts were set to 60 seconds. Sixty. As if the system was saying „hey, take your time, no rush, the user will wait.“ And the real kicker? Any motivated customer could bring the whole thing down just by sending a bunch of requests. The system had zero headroom because every single request was already crawling through molasses.
So I did what any reasonable engineer would do. I got angry, opened my laptop, and started figuring out why.
you can’t fix what you can’t see
Here’s the thing about slow APIs. Everyone has opinions. „It’s the database.“ „It’s the network.“ „It’s that one service Dave wrote three years ago.“ Cool. Nobody actually knows. They’re all guessing.
The first real move is distributed tracing. Not some half-assed logging where you grep through a million lines hoping to spot a timestamp that looks suspicious. Actual tracing. OpenTelemetry spans that follow a request from the moment it hits your load balancer to the moment a response leaves your system.
You create spans for each service the request touches. Server side, client side, it doesn’t matter. You want to see everything. The API gateway, the business logic, the database call, the message queue, the external payment provider, all of it stitched together in one trace. When you look at a waterfall view of a 17 second request and you see that 14 of those seconds are spent waiting on a synchronous call to some audit service… you don’t need a meeting to figure out what to fix. The trace screams it at you.
This is where most teams stop, by the way. They set up tracing, find one bottleneck, fix it, and move on. That’s a mistake. You need the full picture.
metrics and profiling: going deeper
Distributed tracing tells you where the time goes across services. But what about inside the service itself? That’s where metrics and profiling come in.
Instrument your code. Measure how long your own operations take versus how long you spend waiting on external systems. This distinction matters more than people realize. If your service spends 800ms doing CPU-bound work, that’s a profiler problem. Fire up your language’s profiler, find the hot path, and optimize it. Maybe you’re serializing something unnecessarily. Maybe you’re doing a loop where you should be doing a lookup. The profiler will tell you.
But if your service spends 800ms waiting on a database query, the profiler won’t help. Now you need to look at the query itself. Run EXPLAIN ANALYZE on it. I cannot overstate how many performance problems I’ve seen solved by one engineer finally running EXPLAIN ANALYZE and discovering the query was doing a full table scan because someone forgot an index. Or the query planner chose a nested loop join when a hash join would’ve been 100x faster. The database is not magic. It makes bad decisions sometimes, and you need to check its work.
the quick wins that actually move the needle
After instrumenting everything, patterns emerge fast. And a lot of them come down to the same fundamental mistakes teams keep making.
stop making synchronous calls to external systems
This is the big one. The single most impactful change you can make to your API latency.
Your synchronous API should make exactly one external call: to your database. That’s it. Everything else, the payment provider, the audit service, the notification system, the analytics pipeline, the other microservice that „just needs a quick check“, all of it needs to be asynchronous.
There are two reasons for this and both are critical. First, every synchronous external call adds its latency directly to your response time. If your database takes 20ms and your payment call takes 400ms and your audit call takes 300ms, congratulations, your API now takes 720ms at minimum. And that’s assuming nothing goes wrong. When that payment provider has a bad day and starts responding in 3 seconds instead of 400ms, your API is now a 3.3 second API. Your users didn’t sign up for that.
Second, and this one is more subtle, synchronous calls to multiple systems create the dual write problem. You update your database, then call the payment system. The payment call fails. Now what? Your database has one state, the payment system has another, and you’re stuck writing compensation logic that will haunt your codebase for years. Push it to a queue. Let the consumer handle retries with proper idempotency. Sleep well at night.
paginate your database queries
I’ve seen production APIs that selected entire tables. Not „SELECT * FROM users WHERE…“ with a reasonable filter. Just „SELECT * FROM users.“ In production. With 2 million rows.
The query itself might technically complete in a few seconds. But then you’re serializing 2 million objects, shoving them through the network, parsing them on the application side, and probably mapping them to some domain object one by one. Your memory spikes, your garbage collector panics, and your response time goes through the roof.
Paginate everything. Use cursor-based pagination if you can, offset-based if you must. Set sensible defaults. No API should ever return more than a few hundred records in a single response unless there’s a very specific reason for it.
cache the expensive stuff
This one seems obvious but I keep seeing teams skip it because „we want real-time data.“ You know what users want more than real-time data? A response. They want the page to load.
If you have an operation that takes 2 seconds because it aggregates data from five different sources, and that data changes maybe once an hour, cache the result. Redis, Memcached, even an in-memory cache if the dataset is small enough. A cache hit at sub-millisecond latency versus a 2 second computation is not a tradeoff. It’s a no-brainer.
The key is knowing what to cache and for how long. Not everything needs to be cached. Not everything can be. But expensive, frequently accessed, slowly changing data? Cache it yesterday.
precompute what you can
This is the evolution of caching. Instead of computing something expensive on demand and then caching the result, just compute it ahead of time.
Do you really need to make 10 database queries and 3 external API calls every time a user opens their dashboard? Probably not. Most of that data doesn’t change between requests. Run the computation asynchronously on a schedule, or trigger it when the underlying data changes, and store the result. When the user requests their dashboard, you read a single precomputed document. Fast. Simple. Boring in the best way.
I’ve seen teams turn 4 second dashboard loads into 50ms reads by switching from on-demand computation to precomputed views. That’s not a small improvement. That’s a different product.
stop returning data you don’t need
This one is so simple it’s almost insulting. But I see it everywhere. An API that returns the entire user object with 47 fields when the client only needs the name and email. A list endpoint that returns every column from a joined query including blobs, audit trails, and nested relationships three levels deep.
Over-fetching kills you in two places. First, the database has to read, join, and transfer more data than necessary. Second, your application has to serialize all of it into JSON, which for large payloads is not cheap. I’ve profiled APIs where JSON serialization alone accounted for 30% of the total response time. Thirty percent. Just turning objects into strings.
Select only the columns you need. Return only the fields the client asked for. If you’re building a REST API, consider supporting sparse fieldsets or just building more focused endpoints. If you’re on GraphQL, well, this is supposed to be the one thing GraphQL is good at. Use it.
set aggressive timeouts and circuit breakers
Remember those 60 second timeouts I mentioned? That’s not a timeout. That’s a prayer. A real timeout should reflect how long your users are actually willing to wait. If your SLA is 500ms, your timeout to any downstream dependency should be well under that. Something like 200-300ms with a retry budget.
And when a dependency starts failing, stop calling it. Circuit breakers exist for exactly this reason. If your payment service is timing out, hammering it with more requests doesn’t fix the problem. It makes it worse. Open the circuit, return a degraded response or queue the work for later, and let the failing system recover. Libraries like Hystrix, resilience4j, or Polly make this straightforward. There’s no excuse for not having circuit breakers in 2026.
Without them, one slow dependency takes down your entire system. With them, you get graceful degradation instead of cascading failure. That’s not just a latency improvement. That’s the difference between a bad hour and a full outage.
compress your payloads
If your API returns large JSON responses and you’re not compressing them, you’re wasting bandwidth and time for no reason. Gzip or Brotli compression on API responses can shrink payload sizes by 70-90%. That’s less data over the wire, which means faster transfers, especially for clients on slower connections or mobile networks.
Most web frameworks and reverse proxies support this out of the box. Nginx can handle compression at the edge so your application doesn’t even need to think about it. Enable it. It’s practically free performance.
choose the right protocol
If all your internal services are talking REST over HTTP/1.1 with JSON serialization, you’re leaving performance on the table. gRPC with Protocol Buffers over HTTP/2 is faster in almost every measurable way. Smaller payloads. Binary serialization. Multiplexed connections. Built-in streaming.
REST is fine for public APIs where developer experience and discoverability matter. But for service-to-service communication inside your infrastructure? gRPC is the move. I’ve seen teams cut their internal call latency by 30-40% just by switching protocols. No algorithmic changes. No architecture redesign. Just a more efficient wire format.
going deeper: infrastructure-level optimizations
Everything above will get you from „embarrassingly slow“ to „reasonably fast.“ But if you want to push into genuinely low latency territory, you need to think about infrastructure.
co-locate your app and database
Put your application server and your database in the same availability zone. Not just the same region. The same AZ. A cross-AZ round trip in AWS is typically 1-2ms. That sounds small until you realize your API makes 5-10 database calls per request. Now you’re adding 10-20ms just from network hops between zones. In the same AZ, that drops to sub-millisecond.
Yes, this means you need to think about your high availability strategy differently. Use read replicas in other AZs for failover. But your primary write path and your hot read path should be as physically close together as possible.
And the bonus? Cross-AZ data transfer costs money. AWS charges for it. So you’re saving latency and reducing your cloud bill at the same time. That’s the kind of optimization that makes finance teams smile.
put your infrastructure near your users
If your users are in Europe and your servers are in us-east-1, physics is working against you. The speed of light is fast but it’s not instant. A round trip from Frankfurt to Virginia is roughly 80-90ms just for the network. Before your server even starts processing the request, the user has already waited almost 100ms.
DNS-based geo-routing solves this. Route users to the nearest deployment. If you’re running in multiple regions, you can use Route 53 latency-based routing, Cloudflare’s geo-steering, or whatever your DNS provider offers. Combined with edge caching for static content, this can dramatically reduce perceived latency for users far from your primary data center.
connection pooling and keep-alives
Opening a new TCP connection for every request is expensive. TLS handshakes are even more expensive. Use connection pooling for your database connections. Use HTTP keep-alive for your internal service calls. Reuse connections aggressively.
I’ve seen apps where every single database query opened a new connection, did the query, and closed the connection. The connection overhead was costing more than the query itself. A properly configured connection pool turned 40ms queries into 5ms queries overnight.
async I/O and concurrency
If your API needs to fetch data from three independent sources, don’t call them sequentially. Call them concurrently. If each takes 100ms, sequential execution gives you 300ms. Concurrent execution gives you 100ms. Same data, same sources, one-third the time.
Most modern frameworks support this natively. Use asyncio.gather() in Python. Use Promise.all() in Node. Use goroutines in Go. If your language supports concurrent I/O, use it. There is no reason to wait for something you don’t depend on.
optimize your serialization
JSON is human-readable. It’s also painfully slow to serialize and deserialize compared to binary formats. If you’re doing service-to-service communication, you’ve already heard me say gRPC. But even on the application side, your choice of JSON library matters more than you think.
In Java, switching from Jackson with default settings to a properly configured instance, or to something like DSL-JSON, can cut serialization time in half. In Python, orjson is roughly 10x faster than the standard json module. In Go, sonic or easyjson blow the standard library out of the water for large payloads. These are drop-in replacements in most cases. Five minutes of work for a measurable latency improvement on every single request.
use read replicas for read-heavy endpoints
If 80% of your traffic is reads and you’re routing all of it to your primary database, you’re bottlenecking yourself for no reason. Spin up read replicas and route read-only queries to them. Your primary handles writes. Your replicas handle reads. The write load drops, query performance improves across the board, and you get horizontal read scalability basically for free.
Yes, there’s replication lag. For most read endpoints, a few hundred milliseconds of staleness is completely fine. Your users will not notice that their dashboard data is 200ms behind. They will absolutely notice if the page takes 4 seconds to load because the primary database is drowning in read queries.
lazy load and defer what you can
Not every piece of data in your API response needs to be fetched upfront. If your endpoint returns a user profile with their recent orders, notification count, and recommendation feed, does all of that need to load before you send the response?
Return the core data immediately. Let the client fetch supplementary data in parallel or on demand. This is especially effective for APIs that power UIs. The user sees the important stuff instantly while the less critical pieces load in the background. Perceived performance matters just as much as actual performance.
rate limiting and backpressure
This isn’t strictly a latency optimization but it protects your latency from getting destroyed. Without rate limiting, one aggressive client can saturate your API and spike latency for everyone else. I mentioned earlier that customers could bring down our system by just sending a lot of requests. That’s because there was zero backpressure.
Implement rate limiting at the API gateway level. Use token bucket or sliding window algorithms. Return 429s before the request even hits your application. And internally, use backpressure mechanisms. If your service can process 1000 requests per second and it’s receiving 5000, the right move is to reject the excess early, not let them pile up in memory until your pod OOMs and restarts.
The fastest request is the one you don’t process at all.
the mindset shift
Building fast APIs isn’t about one silver bullet. It’s about a hundred small decisions that compound. Every synchronous external call you remove saves hundreds of milliseconds. Every paginated query prevents a disaster. Every cached result eliminates redundant computation. Every co-located service shaves off a few more milliseconds.
The teams that build genuinely fast systems aren’t smarter than everyone else. They just measure obsessively and refuse to accept „it’s fine“ when the traces say otherwise. They treat latency as a feature, not an afterthought. And they understand that a 2 second API isn’t just slow. It’s a system that’s one bad day away from falling over entirely.
We went from a p99 of 2 seconds to under 200ms across the board. The 50 second endpoints? Gone. Replaced with async workflows and precomputed results. The 17 second ones? Turned out to be unindexed queries hitting tables that had grown 10x since anyone last looked at them.
None of this was rocket science. It was just the boring, methodical work of measuring, understanding, and fixing. One span at a time.
Peace, nerds.