search

sw33tLie / bbscope

Scope gathering tool for HackerOne, Bugcrowd, Intigriti, YesWeHack, and Immunefi!
Apache License 2.0
1.01k stars 145 forks source link

Handle 429 errors due to rate limiting on H1 API #13

Closed h1pmnh closed 2 years ago

h1pmnh commented 2 years ago

The H1 API has a built-in rate limit (currently set at 600 requests per minute). If you have a large number of programs, you will hit this rate limit and it will cause some portion of your programs gathered by bbscope to falsely report NO_IN_SCOPE_TABLE when in fact what has happened is the API returned a 429 response due to rate limiting.

This change adds a simple loop with 5 second delay (up to 50 seconds) so that the total requests comes under the rate limit and full program details can be retrieved.

As I am learning Golang it is possible some of this may not be idiomatic and you are welcome to edit as necessary :)

h1pmnh commented 2 years ago

Please note you will only see this error if you have a large number of programs such that you are exceeding the rate limit of the API. Many users will not encounter this limit. An alternative fix for this would be to reduce (or make configurable) the number of threads or add a delay between requests.

sw33tLie commented 2 years ago

Hey there, thank you for sending this PR. This has been on my to-do list for a while but couldn't find time to add it yet. Merging now, appreciate your help :)