owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
72 stars 20 forks source link

walkthrough: Chart revision tool fails to submit revisions #1175

Closed pabloarosado closed 1 year ago

pabloarosado commented 1 year ago

One-liner

Chart revision tool fails to submit revisions.

Environment

Context & details

On staging, I'm trying to map variables from the Electricity Mix (2023) to the Electricity Mix (2023b). Everything goes well until I press the button to submit revisions, and then I get the error:

File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
File "/Users/prosado/Documents/owid/repos/etl/walkthrough/charts_v2/__main__.py", line 89, in <module>
    push_submission(submission_config, owid_env)
File "/Users/prosado/Documents/owid/repos/etl/walkthrough/charts_v2/submission.py", line 105, in push_submission
    comparison = create_chart_comparison(chart.config, config_new)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/prosado/Documents/owid/repos/etl/etl/chart_revision/v2/core.py", line 217, in create_chart_comparison
    if config_1["id"] != config_2["id"]:
       ~~~~~~~~^^^^^^

Screen Shot 2023-06-02 at 09 53 49

lucasrodes commented 1 year ago

The config of chart 5635 is missing the id field. I manually added it to the DB. Could you re-run the chart update and see if it works?

In case this happens again, I found this out using the error message itself and the terminal log, in particular the following line:

2023-06-02 11:19.02 [info     ] chart_revision: creating comparison for chart 5635

This line mentions the last chart processed (i.e. likely the one producing the error).

pabloarosado commented 1 year ago

Thanks for looking into this @lucasrodes. I was working on staging and the problem persists. What exactly is the modification you need to do to the problematic chart config?

pabloarosado commented 1 year ago

Ah, I see, I checked the log and realised that it was chart 6356, which didn't have a "id" field in the config. I simply added "id": 6356 at the beginning of the chart config and now it works.