search

quintel / mechanical_turk

Automatic tester for etengine
1 stars 0 forks source link

Improvements for energy flows (overview) data export #169

Closed mabijkerk closed 3 months ago

mabijkerk commented 1 year ago

The data export "Energy flows (overview)" presents the energy flows in the model in a format that is aligned with the Sankey diagram. There are a number of issues with this data export, which this PR aims to tackle.

This PR includes:

Goes with: https://github.com/quintel/etsource/pull/2869

The config of the Sankey is defined here. The config is presented in a readable Documentation of Energy flows (overview) data-export.xlsx. The main changes (page 1) and notes on potential differences (page 2) are described in the Log.docx.

mabijkerk commented 1 year ago

@noracato Semaphore indicates that this PR fails with the message that a number of queries are missing. However, when I run the test locally, it passes. Moreover, I can find the queries that it says are missing on master, and I can also query them in the beta engine Sandbox. Could you look into this?

For example, this is one of the queries that is 'missing: turk_sankey_primary_demand_of_hydrogen_flows

noracato commented 1 year ago

After a rerun, most queries seem to be doing fine. But another issue arises: mechanical turk now takes over 1 hour to run. Semaphore automatically stops runs after 1 hour to protect you from accidentally running never-ending jobs and paying a lot for it.

We can:

@ChaelKruip, any thoughts?

mabijkerk commented 3 months ago

I upped the MT limit, but I'll throw away this PR as it is rather outdated and better tests can be developed for the Sankey.