search

quintel / etmodel

Professional interface of the Energy Transition model.
https://energytransitionmodel.com/
MIT License
33 stars 14 forks source link

http://monitor.et-model.com/ is not transparent #576

Closed ChaelKruip closed 12 years ago

ChaelKruip commented 12 years ago

It is unclear what the allowed error margins are (that make a result turn red or green).

Suggestion: Show the deviation as a percentage. Suggestion: Explicitly show the tolerance.

dennisquintel commented 12 years ago

It's now supersimple: 1% deviation is allowed.

How would you show that percentage? (it's already a bit crammed, the view)

ChaelKruip commented 12 years ago

Maybe a third column? Would look fine IMO.

Perhaps a related suggestion/question:

Why use the same tolerance for wildly different variables? Should this not reflect the expected deviation in the variable somehow?

dennisquintel commented 12 years ago

Why use the same tolerance for wildly different variables?

Simplicity.

Should this not reflect the expected deviation in the variable somehow?

I'm open to that. But like you say: How? Perhaps the user can define the allowed deviation (in %?) for each gquery?

ChaelKruip commented 12 years ago

Sounds good to me. In that case the colors would become more meaningful.

dennisquintel commented 12 years ago

@ChaelKruip: could you give me appropiate allowed deviations to start with?

dennisquintel commented 12 years ago

@ChaelKruip: could you give me appropiate allowed deviations to start with?

ChaelKruip commented 12 years ago

This is likely a complex and time-consuming task:

There are two ways (that I can think of) to obtain the desired deviations:

  1. Calculate them: for each monitored variable, find the functional dependence on other variables/parameters and use error propagation calculations to arrive at the allowed tolerance. Given a variation in each of the relevant parameters, the answer follows directly.
  2. Compute them: Use the ETM to estimate the tolerance by putting in extremal values for the relevant variables/parameters and look at the deviations in the final result (the variable of which we want to determine the deviation).

The first approach is better in principle as it takes into account all (possible) errors in all variables at once in a analytical framework. In the case of complex dependences (merit order calculation for example) this is no longer feasible, however, as there is no clear functional dependence (it is algorithmic essentially). For simple dependencies it might be the best option IMO.

The second approach is applicable to most variables. Its major shortcoming is lack of robustness. One would have to create many instances of the model (varying the errors for all relevant variables in a Monte Carlo sense) to estimate the range of error in the final variable. This is computationally intensive en takes time to implement correctly.

If we really want this, we should discuss a plan of attack but I feel that further development of e.g. the Mechanical Turk will yield the more progress on the testing front.

dennisquintel commented 12 years ago

Closing. Superceded by the Mechanical Turk