As a project manager I would like to check if we are integrating correct data from all e-mobility DCs

[rcavaliere]

We may have an issue with the data integration process from the e-mobility Data Collectors (https://github.com/noi-techpark/bdp-commons/tree/main/data-collectors/emobility-echarging).

It seems that the charging stations do not have variations if we look at time series, e.g. https://analytics.opendatahub.bz.it/#%7B%22active_tab%22:0,%22height%22:%22400px%22,%22auto_refresh%22:false,%22scale%22:%7B%22from%22:1635721200000,%22to%22:1647558000000%7D,%22graphs%22:%5B%7B%22category%22:%22E-Mobility%22,%22station%22:%22ASM_00000131%22,%22station_name%22:%22BZ_V_VOLTA%22,%22data_type%22:%22number-available%22,%22unit%22:%22%22,%22period%22:%22600%22,%22yaxis%22:1,%22color%22:3%7D%5D%7D

We should collect evidences that we exactly collect these data from the sources, so to address this issue with the data providers

[dulvui]

I saw that on analytics testing machine other values are shown: https://analytics.opendatahub.testingmachine.eu/#%7B%22active_tab%22:0,%22height%22:%22400px%22,%22auto_refresh%22:false,%22scale%22:%7B%22from%22:1653170400000,%22to%22:1653775200000%7D,%22graphs%22:%5B%7B%22category%22:%22E-Mobility%22,%22station%22:%22ASM_00000131%22,%22station_name%22:%22BZ_V_VOLTA%22,%22data_type%22:%22number-available%22,%22unit%22:%22%22,%22period%22:%22600%22,%22yaxis%22:1,%22color%22:0%7D%5D%7D They seem more correct than the production values. I will check now the code of the data collector to see what the problem is and I already found some bugs

[rcavaliere]

@dulvui interesting, the same station gives on production the following: https://analytics.opendatahub.bz.it/#%7B%22active_tab%22:0,%22height%22:%22400px%22,%22auto_refresh%22:false,%22scale%22:%7B%22from%22:1653170400000,%22to%22:1653775200000%7D,%22graphs%22:%5B%7B%22category%22:%22E-Mobility%22,%22station%22:%22ASM_00000131%22,%22station_name%22:%22BZ_V_VOLTA%22,%22data_type%22:%22number-available%22,%22unit%22:%22%22,%22period%22:%22600%22,%22yaxis%22:1,%22color%22:0%7D%5D%7D

At least, I see some changes in the time series, but let's be sure that we ingest the data correctly...

[dulvui]

The problem should be fixed now. The was a logical error in the Data Retriever class that the stations got fetched only once on startup and then never again, so the data stayed the same until the application was restarted or deployed again. I'll check now with my data-quality python script, if the data is correct now, but a first test showed no errors (for the first time ever :smile: )

[rcavaliere]

@dulvui good to hear that you could be able to find and fix the problem!

[dulvui]

The data is still correct and analytics.opendatahub.com shows now much more changes for number-avaiable.