This issue is to create an alternative to the current ghsync command that will use the github API list endpoints, instead of the individual endpoints.
The objects retrieved from the list endpoints contain less fields, but that will be enough data for many of the main charts.
keep all the code that is already in place, in case in future versions (or for EE) we want to sync the extended individual details
Create a new sub command for the new functionality (a name related to shallow download, or list endpoints)
New code that does not use the queue mechanism. Synchronous code that waits for each endpoint processing before moving on to the next one.
For resuming support. We need some support for resume, not as a main feature, but as resilience measure. We will rely on the DB to know if a resource needs to be downloaded or skipped.
For example, if the organization “bblfsh” exists, it means we successfully downloaded all the related “child” resources (repositories etc).
The code will INSERT on the DB only when all the “child” resources have been successfully processed and saved.
For termination support: Since we are not in an infinite queue job wait, the process can exit when the main organization loop finishes, as everything is done synchronously
This issue is to create an alternative to the current ghsync command that will use the github API list endpoints, instead of the individual endpoints. The objects retrieved from the list endpoints contain less fields, but that will be enough data for many of the main charts.