bitcointranscripts / transcription-review-backend

7 stars 11 forks source link

Enhance Discord Bot Notifications for Transcript Queuing #256

Closed nully0x closed 9 months ago

nully0x commented 10 months ago

This PR addresses #250 and #253 since they are minor update

nully0x commented 9 months ago

I addressed possible bottleneck we might run into while queueing and sending alerts for large data in this commit

Initially, while queuing transcripts there was no issue with commits < 5 but as we are getting to merge I did performance testing and noticed processing commits > 7 takes ~10sec which means we won't be able to queue up to 10 transcripts at a go (pushEvent) because GitHub webhook expects to get a response while processing in 10sec, to which the implementation timed out for 10 commits while testing benchmarked at 15sec (varies on the content of the files in the commit).

before image

now image

by this, we should be able to process ~ 40 commits in one push request.

The second issue was sending alerts for the commit actions in the push event. Discord raises a 429(surpassing the rate limit when sending up to 10 alerts at the same time) but does not stop sending whichch leads to a 403(unauthorized) error. This won't allow us to know what is wrong with subsequent commits in the same push event.

To mitigate this, I implemented a bottleneck to delay sending each request per 3 seconds to avoid running into rate limiting with discord

To test: please run yarn install to refresh your packages.