Data integrity should be ensured

[ChaelKruip]

Background

There are several ways in which a technology in ETMoses can get its attributes such as capacity, COP, investment_cost etc.

• ETEngine: when opening a scaled scenario in ETMoses, the attributes are imported from ETE
• Profiles: when a new technology is added manually, its profile can contain default attributes (volume or capacity) which are used to initialise it
• Database: when a new technology is added manually, and no profile defaults are present, the attributes are looked up in the database

  Problem(s)

There are several problems with this

• If ETE (ETSource) is updated, the numbers in the profile defaults and database will become inconsistent
• Profiles with (nonsensical) defaults can be added by any user. These profiles will be assigned to technologies and can impact their behaviour erroneously

  Actions

I foresee several possible actions to improve the situation

Short term

• Include some validation checks between ETE and the database
• Distinguish between profiles which can be selected automagically by the app (i.e., when changing concurrency etc.) and those that can only be added manually (by the user).

  Longer term

• Import attributes from ETE exclusively (also when adding technologies manually). This would remove the dependency on the database except for technologies which are not in ETE (buffers, neighbourhood batteries)
• Perhaps use the node.ad files from ETSource directly as the only source for attributes to ensure complete consistency

@dennisschoenmakers @antw do you have any thoughts on this?

[antw]

Distinguish between profiles which can be selected automagically by the app (i.e., when changing concurrency etc.) and those that can only be added manually (by the user).

I like this. :+1:

Import attributes from ETE exclusively (also when adding technologies manually). This would remove the dependency on the database except for technologies which are not in ETE (buffers, neighbourhood batteries)

I also agree with this. When we import a scenario from ETE, we should probably save all of the technology data it sends and use that as the source for default values.