Open pietroalbini opened 3 years ago
The next crates.io deploy (happening in the next few days) will include the following changes to the database dumps:
textsearchable_index_col
column will be removed from crates.csv
, as that column is an implementation detail of crates.io's search. Users importing the database dumps into a PostgreSQL database will not be affected by this change, as a trigger will populate that column at import time.version_downloads.csv
file will only include the last 90 days of data instead of full day-to-day historical data. Cumulative download counts are still available in crates.csv
and versions.csv
.version_authors.csv
file will be removed, as that data was deleted from the crates.io database too.We also plan to make the following changes in the future:
version_downloads.csv
will be moved out of the database dump into separate files, one for each day. This will allow clients interested in this data to download it separately.Two relevant changes were just deployed:
badges
tablecrate_downloads
table, which is supposed to replace the crates.downloads
column soon. this was done for performance reasons to reduce the amount of bloat in the crates
table from the regular downloads
column updates. at the moment the data should be in sync, but if everything works out we will stop writing to the crates.downloads
column in the near future and eventually remove it.crates.downloads
column. we will keep the column around for now to avoid unnecessary schema churn, but once the system has shown the expected performance benefits we will most likely remove the column completely.crates.downloads
column. please us the crate_downloads
table instead.default_versions
table with a mapping from crates to their "default" version, that will be shown by the frontend and used in e.g. reverse dependency queries.
This is a low-traffic issue tracking all the changes happening to the experimental database dumps. We recommend subscribing to this issue to get notified whenever we make some changes to the contents of the dumps.