cmu-delphi / delphi-epidata

An open API for epidemiological data.
https://cmu-delphi.github.io/delphi-epidata/
MIT License
100 stars 62 forks source link

Update Google Docs Meta Data #1451

Closed github-actions[bot] closed 1 month ago

github-actions[bot] commented 1 month ago

Updating Google Docs Meta Data

sonarcloud[bot] commented 1 month ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

melange396 commented 1 month ago

New columns added (in alphabetic order) :

Additionally, some orderings of the comma-separated values under column Available Geography have changed, but this should be inconsequential.

melange396 commented 1 month ago

above items verified with this code:

import pandas as pd

base_url = 'https://github.com/cmu-delphi/delphi-epidata/raw/{}/src/server/endpoints/covidcast_utils/db_signals.csv'

current = pd.read_csv(base_url.format('dev'), na_filter=False)
proposed = pd.read_csv(base_url.format('bot/update-docs'), na_filter=False)

# this code assumes columns have not been removed or renamed,
# and that no new rows have been added or had their ordering changed

new_cols = set(proposed.columns) - set(current.columns)
print("new cols:", sorted(new_cols))

non_matching = (proposed[current.columns] != current)
diffs_per_col = non_matching.apply(sum)
print(diffs_per_col)
# => 400, only in 'Available Geography'

# produce alpha-sorted "Available Geography" from each csv for accurate comparison purposes:
current_ag_norm = current['Available Geography'].apply(lambda g: ','.join(sorted(g.split(','))))
proposed_ag_norm = proposed['Available Geography'].apply(lambda g: ','.join(sorted(g.split(','))))
diff_geos = (current_ag_norm != proposed_ag_norm)
print("Number of rows with different geographies:", sum(diff_geos))
# => 0
melange396 commented 1 month ago

i dont know what i did, but the history is screwed up... im going to close this pr, delete the branch, and try the GH action again.