search

ActiveConclusion / COVID19_mobility

COVID-19 Mobility Data Aggregator. Scraper of Google, Apple, Waze and TomTom COVID-19 Mobility Reports🚶🚘🚉
MIT License
267 stars 58 forks source link

Google: report format changed #16

Closed ActiveConclusion closed 4 years ago

ActiveConclusion commented 4 years ago

Google added in a recent update two columns:

  1. iso_3166_2_code
  2. census_fips_code

I'm not sure that these attributes are useful, but some fixes needed to the script.

Also, Google added more regions.

ActiveConclusion commented 4 years ago

There are two significant changes in the last update (2020-07-27) of Google reports:

  1. Added metro_area column. It's one more administrative level, for metropolitan areas.
  2. Before this update, the second level of subregions (column sub_region_2) was used only for the US counties. Now, there are many countries for which data on this level were added.

Currently, I exclude data for the second level of subregions and metropolitan areas in this preprocessed Google Mobility report.

I think to add an extended version of the report with new data, but I haven't decided yet how to do it better.

ActiveConclusion commented 4 years ago

The raw Google Mobility report is stored in the ZIP archive in this repo now due to the large file size.

ActiveConclusion commented 4 years ago

After some time of reflection, I decided to make a breakdown of the data by geographic principle. So, in the recent update, I added detailed reports for Brazil (a lot of data is available for them) and for the world regions (Europe, Asia+Africa, North and South America + Oceania). You can check new data here. Also, I decided to merge the "metro area" column with "sub_region_1" because I don't see any advantages of having this column separately. As Apple and Waze do not have that detailed data, I have no plans to merge the new Google data with them (I still haven't merged Waze data :scream:)

I leave this issue open for some time for suggestions/comments about it.