Open RyanCavanaugh opened 3 years ago
Thanks for the report @RyanCavanaugh ! That seems bad! I will dig in to this and see what is going on.
I am able to repro this:
./bin/run search:issues --token <token> --owner microsoft --repo typescript --format JSON --since=2020-06-01 > typescript.json
Looks like the GraphQL query gets to page 10 and then can't find the next page (hasNextPage
is false). Debugging this to see what is happening.
The Search API is documented to only return the first 1,000 results
I've written a similar tool for bulk export and AFAIK the only alternative is to go through the Issues graph instead of the Search graph.
I opened #45 to discuss how we want to solve this issue, since it will require a bit of re-architecting.
I ran this on microsoft/TypeScript and was surprised to see only a 1.6 MB file produced, since I know we have much more content than that. The generated JSON, though, only includes 1,000 issues, so it misses the other ~95% of the issues to have crossed our repo.