search

signalstickers / Adhesive

A chatbot serving as your glue between Telegram and Signal sticker packs
GNU Affero General Public License v3.0
95 stars 11 forks source link

The rate limits suck #4

Open ioistired opened 3 years ago

ioistired commented 3 years ago

Conversion takes 9–20 minutes right now. This is unacceptable.

In the short term, concurrency levels need to be tuned in order to maximize rate limits. The concurrency levels would also ideally be based on a formula that factors in the per-account rate limits and the number of concurrent command invocations.

In the long term, discussion needs to happen with the signal team to work on some sort of private rate limit exemptions.

ioistired commented 3 years ago

Signal and Telegram both have clients first upload each individual image, then the pack metadata. Signal adds an additional request at the beginning to obtain credentials for uploading each image,

On the Signal side, on the level of one pack, the rate limit is 50 packs + an additional pack every 72 minutes (20 per day). The code that limits (or not?) individual image uploads is not public. In fact, it doesn't hit an API endpoint at all, but a CDN. So empirical testing is needed to determine the rate limits on CDN uploads, if any.

On the Telegram side, individual image uploads definitely do get "flood waited" so research is needed to see what those limits are. Pack creation is probably rate limited too, but image uploads are more likely the bottleneck here.

derhagen commented 3 years ago

I usually get an error message like this, independet of the sticker pack: An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 16121611191210955099

It comes up pretty much immediately, though. Is this due to some rate limit or an unrelated bug?

ioistired commented 3 years ago

@derhagen, it's probably rate limits, but I can't tell as I only keep the past 7 days of logs and I didn't get to your report until now.

derhagen commented 3 years ago

@iomintz Here we go again: An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 11512793617713579862

ioistired commented 3 years ago

Yeah that's rate limit exceeded. At the moment, whether Signal will allow you to run the command is kind of a lottery. While I work something out, you can alleviate the issue by running your own instance, and there's also a CLI planned so you don't have to set up signald to use it as well.

derhagen commented 3 years ago

I didn't get where the rate limit comes into play. Is it when handing the new sticker pack to signal and generating a signal.art link? If so, maybe, the pull request to signalstickers could be created anyway.

ioistired commented 3 years ago

Is it when handing the new sticker pack to signal and generating a signal.art link?

Yes

If so, maybe, the pull request to signalstickers could be created anyway.

No, as signalstickers.com requires a signal.art link.

rugk commented 1 year ago

An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 3708815344375093597

Zerogoki00 commented 2 months ago

An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 3708815344375093597

I have the same problem