Scaled scenarios bypass Input cache

[ploh]

For scaled scenarios, the Input.cache is always bypassed. This might impact performance and it also leads to wrong results:

When warming up the Input.cache for a normal scenario, this is done based on a temporary "blank" scenario without any user_values.

This is important in case of an Input whose start_value_gql ultimately depends on other Inputs' values, e.g. households_solar_pv_solar_radiation_market_penetration.ad (which depends on households_potential_electricity_production_of_solar_roof_pv.gql and this on residences_roof_surface_available_for_pv.ad). In this case the computed start_value still remains independent of the other Inputs' settings - because a "blank" scenario is used instead of the actual one.

I suppose this behaviour for normal scenarios is intended!? Or should the start_value of one Input depend on the current user_values?

I sugget to re-enable the cache for scaled scenarios (and add a spec that ensures this). As far as I can see, it was disabled in etengine@5da10f7d1ef43c9f08.

[antw]

Or should the start_value of one Input depend on the current user_values?

No I don't believe that they should. Unfortunately I don't see how we can enable caching of input values for scaled scenarios since there are an effectively infinite number of possible variations (region, disabled sectors, scaling amount) and we'd likely end up with caches full of unnecessary values.

The impact on performance is non-negligible (150ms for a cached, full-size scenario vs 900ms for an uncached, scaled scenario), but I'm satisfied that this is quite low-impact: it only affects the initial page load and scaled scenarios account for only 5% of all production scenarios created in the past year (and 3% of beta scenarios).

However it is still an interesting observation that user_values affect the input start values in ETE-scaled scenarios; that is certainly not intentional. The good news is that I expect it will be quite easy to suppress execution of inputs on the scenario used to create inputs.json.

Thanks for investigating this, much appreciated! :+1:

[ploh]

However it is still an interesting observation that user_values affect the input start values in ETE-scaled scenarios; that is certainly not intentional.

You probably already noticed, but just to make sure: It is not just a GUI problem, i.e. sliders starting in different positions, but also affects gqueries which directly depend on input_values, e.g. etflex_households_solar_pv_installed. (This is actually how I found out about the problem.)

[ChaelKruip]

@antw what is the status here?

[antw]

I will take another look into this to ensure that it does not affect derived datasets; if it does, I expect I can fix that.

For scaled scenarios I think we might have to live with the problem for the time being. It's very minor, and perhaps ETLocal will end up offering a solution by replacing the scaled scenario system.