search

lzell / AIProxySwift

Client for AIProxy
https://www.aiproxy.pro
73 stars 8 forks source link

Replicate Polling Exemption? #57

Open MartinW opened 1 month ago

MartinW commented 1 month ago

Could polling for Replicate results be exempted from rate limits?

Reasons: The rate limit is to prevent too many paid requests, but polling calls are not charged. The fair use rate limit can be used by polling on some long requests. Increasing the rate limit, might encourage misuse.

lzell commented 1 month ago

@MartinW I like the idea of skipping rate limits for polled requests. We may have a short-term fix for you. Are you using Replicate for Flux image generations? If so, please update to 0.34.0 which has this change: https://github.com/lzell/AIProxySwift/pull/60

MartinW commented 3 weeks ago

Synchronous API in the short-term works well. For longer running jobs, skipping rate limits would be good.