Add a progress bar for data ingestion

tweag / nixpkgs-graph-explorer

Explore the nixpkgs dependency graph

MIT License

15 stars 0 forks source link

Open zz1874 opened 1 year ago

zz1874 commented 1 year ago

Fixes 122

This PR adds a progress bar for data ingestion. After executing poetry run python -m explorer.api.ingest [file], the output will be a progress bar

  [####################################]  [number of derivations ingested/number of derivations]  percentage

instead of

INFO:__main__:/nix/store/.....

GuillaumeDesforges commented 1 year ago

Both the logging (which I'll turn into structured logging at some point) and the progress bar have their own merits:

the logging will help monitoring ingestions (mainly failures and performances)
the progress bar is helpful for running locally when there is no monitoring to be done

Please make it so we can do both using CLI option flags.