TeamNewPipe / NewPipeExtractor

NewPipe's core library for extracting data from streaming sites
GNU General Public License v3.0
1.39k stars 420 forks source link

[tests] Update user agent #1192

Closed TobiGr closed 4 days ago

TobiGr commented 4 months ago

This updates the user agent used for testing purposes to a modern version.

This also modifies the RecordingDownloader to automatically prevent hitting rate limits ("sign in to confirm you are not a bot") + RecaptchaExceptions and handle them if they occur: There is now a max number of requests which are made per minute (default is 30). Requests are postponed if the limit is reached. When running into a RecaptchaException the RecordingDownloader waits between 35 and 60 seconds before retrying the request once again. This slows down the time for generating new mocks to ~5 minutes but prevents faulty mocks.

[!Warning] All YouTube tests passing the real tests also pass the new mock tests. However, the Bandcamp tests are failing locally now. Bandcamp tests run by GH Actions pass however.

TobiGr commented 4 months ago

All tests passing the real tests also pass the new mock tests.