The Challenge of Provider Keys
Upstream AI providers impose per-key credit caps, rate limits, and occasional outages. A single exhausted key can take down your entire application if you don't have rotation logic built in.
SAPI's key pool manager continuously monitors the health and remaining credits of every upstream key in the pool.
Automatic Rotation and Failover
When a key approaches its credit limit or receives a rate-limit response, SAPI automatically routes the next request to a healthy alternative key — often before your users notice any degradation.
If an entire provider experiences an outage, model mappings allow transparent failover to equivalent models on alternate providers where configured.
Real-Time Credit Tracking
Every request updates credit balances in real time. The admin dashboard shows per-key utilization, margin analytics, and activity logs so operators have full visibility into pool health.
This operational transparency is what separates a gateway from a simple proxy.