noi-techpark / bdp-core

Open Data Hub / Timeseries Core
https://opendatahub.com
Other
9 stars 4 forks source link

As an AI expert, I would like to know which type of traffic sensor technology is used by a traffic station, so to best process the associated measurements available #276

Closed rcavaliere closed 3 months ago

rcavaliere commented 7 months ago

CISMA and u-hopper need for their AI model an additional metadata called "sensor_type", so to use this field to differentiate between traffic stations to be considered in their calculations or not. Waiting for the Data Browser to be finalized and be available for such operations, the idea would be to have also in this case something like a CSV file in which to provide (by hand) this kind of flag.

dulvui commented 5 months ago

@rcavaliere I added now the mapping with some test data for the stations A22:674:1 and A22:674:2 image

The csv file is available here: https://github.com/noi-techpark/bdp-commons/blob/main/data-collectors/traffic-a22-elaborations/src/main/resources/sensor-type-mapping.csv

I saw that the station codes are composed by A22:station_id:sensor_id. Should the type be associated the specific station code by also checking the sensor_id, like it is implemented now, or can they be simplified by saying all stations with id 674 have a specific sensor type? So the station mapping of A22:674:1 and A22:674:2 could be replaced by a simple A22:674.

rcavaliere commented 5 months ago

@dulvui you can simplify, since the technology is the same at parent level. If you fix this I can then complete the table

dulvui commented 5 months ago

@rcavaliere I simplified it now and it works. But I saw that it works only with stations that have data, because stations that are not active don;t get synced, and so the metadata doesn't change. If this is a problem, I would have to call the mobility API to get all stations, instead of using the one I get from A22 database.

rcavaliere commented 5 months ago

@dulvui that's fine like this, if we then have a similar requirement then we will find a solution for that. Let me test this before going in production

rcavaliere commented 5 months ago

@dulvui I will try to do this today afternoon after I am back from B.