재시도, 타임아웃, 폴백이 상호작용하기 시작할 때
[원문 본문]
Bart Rozycki
Posted on Oct 28
When retries, timeouts and fallbacks start interacting
# ai
# api
# architecture
# python
You call an LLM API and it times out. So you retry.
The retry gets a 429. You wait and try again. This time the provider responds, but the output fails validation. Maybe you have a fallback provider, so you call that instead.
Now the interesting question is no longer whether you know how to retry an HTTP request.
It is how all of those decisions interact.
This problem shows up constantly in AI systems because model APIs are external, relatively expensive and sometimes unpredictable. Providers have different rate limits, latency profiles and failure modes. A request may also have side effects elsewhere in your system, so blindly repeating it is not always safe.
Retries, timeouts, fallbacks, circuit breakers and rate limits are individually simple concepts. The difficult part is defining what should happen when several of them apply to the same operation.
Should validation failure trigger a retry?
Should a fallback run after a timeout?
What happens if the fallback fails too?
If a provider tells you to retry after 20 seconds but your normal backoff says 2 seconds, which one wins?
If the operation eventually succeeds after four attempts, do you report success or the full sequence of failures that led to it?
Those questions are what led me to build RelPrim.
RelPrim is a Python library for reliability around external operations. It models retries, timeouts, fallbacks, circuit breakers, idempotency, rate limit recovery and validation together rather than treating each one as an unrelated wrapper.
The same model works for LLM providers, payment APIs, internal services or any other external dependency.
It also keeps structured execution information about what happened. If an operation succeeds after several retries or through a fallback, you can inspect that instead of seeing only the final return value.
I have just released RelPrim 1.0.
Most of the work before the stable release was not adding new features. It was tightening cancellation behavior, typing, policy interactions and the public API so that combining these mechanisms stays predictable.
The project is here:
https://github.com/bart-rozycki/relprim
Documentation:
https://bartrozycki.com/relprim
I would be interested in hearing how people building AI systems handle reliability across multiple model providers, especially when retries, rate limits, validation and fallbacks start interacting.
Top comments (2)
Subscribe
Personal
Trusted User
Create template
Templates let you quickly answer FAQs or store snippets for re-use.
Submit
Preview
Dismiss
Collapse
Expand
Paul Irolla
Paul Irolla
Paul Irolla
Follow
Building Tokenade, a free and open-source token optimizer for coding agents (Claude Code, Codex, Cursor). Runs locally.
Location
Nantes
Joined
Oct 8, 2024
•
Oct 29
Dropdown menu
Copy link
Hide
Switching to another model after a failure does not reset the bill. Providers count tokens differently, so the same prompt can cost more on the fallback.
Like comment:
Like comment:
1 like
Like
Comment button
Reply
Collapse
Expand
오피셜메일
오피셜메일
오피셜메일
Follow
1인기업·1인창업자의 바이브코딩, 브랜딩과 마케팅 실행 과정을 기록합니다.
Location
대한민국
Work
오피셜메일
Joined
Jul 29, 2026
•
Oct 29
Dropdown menu
Copy link
Hide
개별 재시도 래퍼의 순서보다 한 작업의 전체 정책을 보게 해주는 문제 제기가 좋네요. Retry-After가 전체 마감시각을 넘는 상황과 취소가 폴백 진입 직전에 들어오는 상황을 함께 시험해보고 싶습니다. 마지막 성공값뿐 아니라 시도 경로를 볼 수 있다는 점도 원인 확인에 중요해 보입니다.
Like comment:
Like comment:
1 like
Like
Comment button
Reply
Code of Conduct
•
Report abuse
Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink .
Hide child comments as well
Confirm
For further actions, you may consider blocking this person and/or reporting abuse
[스타트업스쿨 페이지 요약]
/ 재시도, 타임아웃, 폴백이 상호작용하기 시작할 때 재시도, 타임아웃, 폴백이 상호작용하기 시작할 때 ( dev.to ) Dev.to AI · 7시간 전 · AI 모델 LLM API의 불확실성을 관리하기 위해 재시도, 타임아웃, 폴백 등 여러 신뢰성 메커니즘이 복합적으로 작용할 때 발생하는 복잡성을 해결하는 Python 라이브러리 RelPrim의 출시와 그 중요성을 다룹니다. 이 글의 핵심 포인트 1 LLM API는 외부 의존적이며 비용이 높고 예측 불가능한 특성을 가짐 2 재시도, 타임아웃, 폴백, 서킷 브레이커 등 개별 메커니즘 간의 상호작용 관리가 핵심 과제임 3 RelPrim은 여러 신뢰성 메커니즘을 개별 래퍼가 아닌 통합된 모델로 관리하는 Python 라이브러리