Gh/storage queue - Githubissues

microsoft / ghcrawler

Crawl GitHub APIs and store the discovered orgs, repos, commits, ...

MIT License

377 stars 90 forks source link

Gh/storage queue #126

Closed geneh closed 6 years ago

geneh commented 6 years ago

Implement Azure storage queue provider. Azure storage queue is more suitable for our use cases than the Service Bus because of the following:

Messages can be leased for up to a week compared to maximum 5 min in Azure Service Bus, thus eliminating a need for timers.
It is faster and cheaper.
Azure Storage Explorer as well as the portal make it much easier to inspect queues' contents.
More reliable (no partitioning related bugs etc).