PolicyEngine / policyengine-us-data

A package to create representative microdata for PolicyEngine US.
https://policyengine.github.io/policyengine-us-data/
1 stars 2 forks source link

Fix data uploading #44

Closed anth-volk closed 1 month ago

anth-volk commented 2 months ago

Fixes #24.

Unfortunately, the endpoint that GitHub provides for uploading data to a release (link) does not allow uploading if a resource already exists at the given asset's name. This is because GitHub envisions each release as a standalone item within a versioning system. This meant that when uploading a dataset that already existed in a given release (e.g., cps_2024.h5), the upload would fail with error 422. Unfortunately, GitHub's "update a release asset" endpoint (link) only exists to update metadata, not replace an existing asset.

Furthermore, the original code did not properly wrap the upload action in the tqdm library.

To get around this issue, this PR does the following:

This PR also adds tqdm to the download function.