search

quintel / mechanical_turk

Automatic tester for etengine
1 stars 0 forks source link

mechanical turk for scenarios? #131

Closed michieldenhaan closed 5 years ago

michieldenhaan commented 5 years ago

Is it possible to create Mechanical Turk tests for specific scenarios?

It would be nice if we can so we get a notification when certain outcomes change. For example in Antwerp we created a scenario to reduce CO2 by 50% in 2030 about a year ago. A client recently opened the scenario and it turned out that the ETM now reports a CO2 reduction of 47% or so due to various changes to the model, migrations etc. (the scenario was made on BETA so the changes we made to the insulation slider migration was the main culprit). We have quite a few important scenarios and adding MT tests for them would be a nice tool to monitor their stability.

antw commented 5 years ago

In the tests, instead of

Turk::Scenario(area_code: 'nl', end_year: 2050)

... use Preset with the scenario ID:

Turk::Preset(12345)

This will create a new scenario with all the same inputs and settings as the original. The scenario is read-only and the input values cannot be changed.