designed_to_flowlatency-budgets.mdx
● 28 lessons · freealexkazos.com ↗

Latency budgets

Where do the milliseconds of a prediction go?

⏱ 9 min☰ 9 steps▶ playground
Budget
100 ms
After that
mail goes unchecked
Email gatewaywaits ≤ 100 msScoring API3 replicasXGBoost3 ms
Email gatewayScoring APIXGBoost
  • requests
  • changed in this step