search

noi-techpark / bdp-commons

GNU Affero General Public License v3.0
2 stars 12 forks source link

On-street parking data quality check #551

Open rcavaliere opened 1 year ago

rcavaliere commented 1 year ago

Data quality check by systems that we need to analyze in the data chain:

dulvui commented 1 year ago

I checked the code and the type mapping of free/occupied seems correct. I will write some python script to compare given and saved values, to understand whats wrong.

For the "Parksensoren Meran - Meran Rennweg Nr.1" I can't find an entry in the database, so it never arrived. It's not so easy to understand why it isn't saved, because the data collector uses MQTT so I can't fetch all stations, but have to wait for a MQTT message of the station. I will let the dc run locally today with a special behavior if a "Parksensoren Meran - Meran Rennweg Nr.1" comes in.

dulvui commented 1 year ago

I checked now for several days and data of the station "Parksensoren Meran - Meran Rennweg Nr.1" never arrives. Also the mapping seems correct. Is there any other website or API where the data is visible, so I can compare the data?

rcavaliere commented 1 year ago

@dulvui thanks for the feedback. No, we don't have information on that for systems, I will ask for further details. Let's put this user story on hold until we get further information from the Data Provider

rcavaliere commented 1 year ago

@dulvui I don't know if this can help, see screenshot of systems sent today. They continue to say that we are wrong with the mapping of the data they sent...

image
dulvui commented 1 year ago

@rcavaliere I checked the dc now again and it seems to work fine. But I saw some of the stations have mvalue "0" or "" (that would be correct) and some have the mvalue set to "occupied" or "free" instead of a 0/1 value. https://mobility.api.opendatahub.testingmachine.eu/v2/flat%2Cnode/ParkingSensor/%2A/latest?limit=200&offset=0&where=sorigin.eq.systems&shownull=false&distinct=true&timezone=UTC So it might be that maybe some elaboration modifies the values afterwards or so. I will continue to investigate the problem and check also the parking elaborations.

rcavaliere commented 1 year ago

@dulvui thanks for the info. Please ensure that there the parking forecasts are not applied there, it shouldn't be. On-street parking data should not be elaborated at the moment

dulvui commented 1 year ago

@rcavaliere The problem with the inverted values was on the side of the OTP mentor API and now it is solved. So only on https://mobility.meran.eu/ the parking sensors showed inverted data.

For the missing station "Parksensoren Meran - Meran Rennweg Nr.1" I added now more logging on the data collector to see if any value for that station arrives trough MQTT. I saw that also "Parksensoren Meran - Meran Rennweg Nr.2" doesn't have data since 2/8/2022