SeitaBV / flexmeasures-entsoe

Getting data relevant for energy flexibility services via ENTSO-E into FlexMeasures.
Apache License 2.0
3 stars 1 forks source link

Correct sensors #20

Closed Flix6x closed 2 years ago

Flix6x commented 2 years ago

This PR updates some units and resolutions of sensors. I guess this is a good opportunity to discuss how we should handle database migrations coming from plugins. Just a proposal: add a CLI command in the plugin, one for each migration?

nhoening commented 2 years ago

Here is one direction to think towards: Each plugin can use its own alembic migrations. The plugin could configure that its table is {plugin_name}_alembic_version, so its history of migrations is not mixed with the one from FlexMeasures. Of course, one needs to be careful with changes. If the plugin authors remove a column FlexMeasures works with, for example, things will break. But right now it seems doable. That would be one step more for plugins to be full projects on their own.

Flix6x commented 2 years ago

Worthwhile to note: this PR also moves from an hourly to a quarterhourly CO₂ signal.

Migration steps for correcting units and resolutions:

update sensor set (unit, event_resolution) = ('MW', interval '15 minutes') where (name='Scheduled generation' and unit='MWh' and event_resolution=interval '1 hour');
update sensor set unit='MW' where (name='Solar' and unit='MWh');
update sensor set unit='MW' where (name='Onshore wind' and unit='MWh');
update sensor set unit='MW' where (name='Offshore wind' and unit='MWh');
update sensor set event_resolution=interval '15 minutes' where (name='CO2 intensity' and unit='kg/MWh' and event_resolution=interval '1 hour');

Migration steps for correcting names:

update sensor set name='Wind Onshore' where (name='Onshore wind');
update sensor set name='Wind Offshore' where (name='Offshore wind');
update sensor set name='CO₂ intensity' where (name='CO2 intensity');

Finally, rerun flexmeasures entsoe import-day-ahead-generation --from-date <however far your history went> to fill in the missing quarterhourly values.