After the implementation of pagination for GitHub and BitBucket API (thanks to the report on #1) i was thinking about using some workers to speedup the scraping process.
My idea is to spawn a worker every 5 pages.
Any idea, suggestion and contribution is well accepted.
The current solution available at paginator spawns a goroutine for each page
After the implementation of pagination for GitHub and BitBucket API (thanks to the report on #1) i was thinking about using some workers to speedup the scraping process.
My idea is to spawn a worker every 5 pages.
Any idea, suggestion and contribution is well accepted.
The current solution available at paginator spawns a goroutine for each page