The Pattern
The failure pattern is consistent: an API integration works in development, passes load testing at modest volumes, ships to production, and then fails intermittently or catastrophically when real traffic arrives. The integration was not wrong. It was incomplete. Four causes account for the majority of these production failures.
Cause 1: No Retry Logic
Network calls fail. Third-party APIs return transient errors. A database behind the API has a brief overload. A deployment causes a 30-second window of 503 responses. These are not permanent failures, they are transient conditions that resolve themselves within seconds. An integration with no retry logic converts every transient failure into a permanent error visible to your users.
The production log pattern: a spike of 500 or 503 errors from the third-party API at 2:15 PM, followed by your application returning errors to users for exactly as long as the third-party issue lasted. With retry logic: the 2:15 PM spike exists in the third-party's logs but is invisible to your users because your integration retried and succeeded on the second or third attempt.
Implement exponential backoff with jitter for retries: the first retry after 200ms, the second after 400-800ms (randomized), the third after 800-1600ms. The randomization prevents thundering herd problems where every client in your system retries simultaneously after a brief outage and overwhelms the recovering service.
Cause 2: Rate Limiting Not Respected
Every production API has rate limits. Most return 429 Too Many Requests with a Retry-After header when you exceed them. An integration that does not read this header and back off appropriately will continue hammering the API with requests that all return 429, consuming your rate limit allowance without making progress, and potentially triggering IP blocks or account suspension.
The production log pattern: a cascade of 429 errors starting at a specific timestamp (when a scheduled batch job starts, when a traffic spike hits a feature that calls the third-party API, when a message queue drains rapidly). The 429s continue until the rate limit window resets.
The fix: read the Retry-After or X-RateLimit-Reset header from 429 responses and wait the specified duration before retrying. For batch operations that call rate-limited APIs, introduce deliberate delays between requests to stay within the rate limit rather than hitting it and waiting.
Cause 3: No Circuit Breaker
A slow third-party API is more dangerous than a failed one. If the API is responding but taking 8 seconds per request, every request to your application that needs the third-party API holds a thread for 8 seconds. Under load, your thread pool exhausts and your entire application stops responding, even for requests that have nothing to do with the third-party integration.
The production log pattern: response times for all endpoints begin increasing at the same time a specific third-party API's response time spikes. Your application's error rate rises even for endpoints that do not use that API, because your thread pool is exhausted by the slow integration calls.
The circuit breaker pattern addresses this: after a defined number of failures or slow responses, the circuit "opens" and requests to the third-party API fail immediately with a cached error rather than waiting for the timeout. After a configurable recovery period, the circuit closes and normal operation resumes. Libraries like Polly (.NET) and opossum (Node.js) implement this pattern with minimal configuration.
Cause 4: Connection Pool Exhaustion
HTTP clients maintain a pool of connections to avoid the overhead of establishing a new TCP connection for every request. If your integration creates a new HttpClient (or equivalent) for every request rather than reusing a shared instance, it exhausts the connection pool and the underlying socket ports on the machine, causing connection failures that look like network errors but are actually resource exhaustion.
The production log pattern: intermittent connection refused or socket timeout errors when calling a third-party API, correlating with high request rates. The integration works at low traffic and fails as volume increases.
The fix: use a single shared HTTP client instance (or use IHttpClientFactory in .NET, which manages client lifecycle correctly). Never create a new HTTP client per request.
Bonus: Queue-Based Buffering for Non-Real-Time Integrations
For integrations that do not need to be real-time, such as sending events to a CRM, logging to an analytics platform, or triggering downstream workflows, buffering through a message queue removes the third-party API from your critical request path entirely. Your application writes to the queue (fast, local, reliable) and a background worker drains the queue at a rate the third-party API can handle. The third-party API's availability and performance no longer affects your application's response time.
FriendsBit has built and fixed API integrations across healthcare, enterprise, and government contexts. If you have an integration that is failing under load and you need a production diagnosis, get in touch.