owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
58 stars 18 forks source link

✨ indicator upgrader: pagination, performance, ui/ux #2859

Closed lucasrodes closed 1 week ago

lucasrodes commented 1 week ago

For large datasets, indicator upgrader is not working as good as it could.

This PR attempts to incorporate pagination to the app.

Code

UX/UI

TODO:

owidbot commented 1 week ago
Quick links (staging server): Site Admin Wizard

Login: ssh owid@staging-site-pagination-indicator-upgrader-3

chart-diff: ✅ No charts for review.
data-diff: ✅ No differences found ```diff Legend: +New ~Modified -Removed =Identical Details Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet ``` Automatically updated datasets matching _weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk_ are not included

Edited: 2024-06-24 08:30:23 UTC Execution time: 15.27 seconds

lucasrodes commented 1 week ago

@Marigold I'm not sure why the baking of the staging server is failing. It looks like the bake is trying to remove some variables for which there are charts... But I am not sure why this occurs here and not on master.

Marigold commented 1 week ago

@lucasrodes that was because of this issue. I ran cd owid-grapher && make refresh on your staging server to get a working copy of prod MySQL.