noi-techpark / bdp-core

Open Data Hub / Timeseries Core
https://opendatahub.com
Other
9 stars 4 forks source link

As Municipality of Bolzano, I would like that Open Data Hub provides me some specific improvements so to better control the distribution of accurate real-time parking data #279

Closed rcavaliere closed 1 month ago

rcavaliere commented 6 months ago

The Municipality of Bolzano is using the Open Data Hub API to feed some new Variable Message Signs (VMS) with real-time parking data. In this way motorists are routed efficiently among the available parking infrastructures. However, there are some issues with the current usage of the API. The provider of this VMS system is not so "flexible" and smart and the Municipality of Bolzano asks if we can shift some of the logic on the Open Data Hub. More in particular the requests are the following:

if (current_timestamp - timestamp of last data record) < 2*period then realtime_data = last_data, otherwise realtime = '--'

At the moment it is sufficient to apply this to the datatype "free". Please however implement this in a generic way so that we can apply this on multiple datatypes associated to the station

if last_data < 10 then realtime_data = 0, otherwise realtime = last_data

ohnewein commented 6 months ago

Should these special API calls be in a special section called AIaaS and run in Microservices separated from the Data Consumer API?

My thinking is to avoid the creation of special API calls for a specific client. Avoid creating an App API like the one we want to remove from the tourism domain.

rcavaliere commented 6 months ago

I think we can offer this API for everybody, as additional API call (e.g. https://mobility.api.opendatahub.com/v2/flat,node/ParkingStation/*/realtime). So we use this request to offer something more to the Open Data Hub customer, generic for everybody. Eventually as a premium service...

sseppi commented 6 months ago

@rcavaliere even if we offer this API for everybody, I would prefer to separate the logic in an external AIaaS microservice as proposed by @ohnewein.

In that way we keep the API as standard and clean as possible. Doing so we will have the API that exposes the data as they are and an API that exposes the elaborated one. This could allow us also to fit better different needs that our customer could have. As for example there could be the use case A that requires that real time data are every 10 minutes and use case B that requires that real time data are every minute. In that case it would be easier to fit both use case, maybe only with a configuration of the micro service or including a parameter in the call.

Moreover, thanks to the experience of the Tourism domain and the customer care, I noticed that, in case of problems or need to change something, it is easier to debug an external specific microservice than the entire core.

dulvui commented 6 months ago

I discussed the issue with Clemens internally and I will implement a custom AIaaS that will handle the requested special features. The new API will be available on a new domain like for example https://parking-bolzano.opendatahub.com

@rcavaliere should this API return directly the data, so that all logic is handled in the service implementation and they can simply call the URL directly? Or should the URL have also optional parameters like for example a station list or data type?

rcavaliere commented 6 months ago

@dulvui thanks for the feedback. No, let's go for a simple URL they have to consume, all the logic is by us.

rcavaliere commented 6 months ago

@dulvui before starting any activities, can you please make an effort estimation for such an implementation? We will then make an agreement with the Municipality of Bolzano, and once obtained, than we can start with the work.

dulvui commented 6 months ago

@rcavaliere Great! Having a single URL makes it a bit easier to implement. I will do an effort estimation today and let you know through email.

ohnewein commented 5 months ago

Waiting for OK by municipality on the estimation.

dulvui commented 3 months ago

Another way to realize this could also be trough a simple elaboration, that creates a new String data type that could be called realtime-info or something similar. The elaboration creates the needed string values and no custom API needs to be implemented. The only needed action by the data consumer is to use the new datatype when doing the API call.

So for example this call would give the free places, if the data is real time and -, if the data is not real time: https://mobility.api.opendatahub.com/v2/flat,node/ParkingStation/realtime-info/latest

rcavaliere commented 3 months ago

@ohnewein @dulvui @clezag the Municipality of Bolzano has formalized the contract with us. We can start with this activity!

dulvui commented 3 months ago

@rcavaliere The first MVP is now online https://parking-bz-realtime.opendatahub.testingmachine.eu/

I also implemented a own "free" calculation, to solve the problem with the real time freshness as they mentioned. The original data-collector only writes occupied, so it could be that the free value, calculated by the elaboration, is 5 minutes behind.

Some questions: 1) What are the exact stations they need? 2) We probably also need to know what fields they use, except mvalue? For now I only provide mvalue and scode.

rcavaliere commented 3 months ago

@dulvui wonderful! As far your questions are concerned:

  1. The same as https://webcomponents.opendatahub.com/webcomponent/parking-dashboard/fullscreen
  2. I think these two values could be enough, but maybe they want something else to recognize the single parking areas. Let me manage this with the VMS supplier of the city, in case I will let you know!
rcavaliere commented 2 months ago

@dulvui can you set up in the meantime also the production URL, i.e. https://parking-bz-realtime.opendatahub.com/?)

dulvui commented 2 months ago

@rcavaliere Yes, now it is working also in production https://parking-bz-realtime.opendatahub.com/

rcavaliere commented 2 months ago

@dulvui I have spoken with the technical counterpart working on the displays.

We need to fix / check following points:

dulvui commented 2 months ago

@rcavaliere I fixed now open issues you mentioned and I can confirm that the data does change. This application forwards the call directly to our mobility API, so the actual data is returned and no caching or similar could return old data. I also added the timestamp now (so they can see the last change of the data) and return the same JSON structure, as the ninja API would return with the fields data, offset, limit etc. as mentioned in the email from this morning.

It is already live in production https://parking-bz-realtime.opendatahub.com/