As a Data Consumer I would like that the VMS data of Autobrennero is much more "real-time"

[dulvui]

I increased now the maximum RAM limit, because currently it ran out of memory after some time. Now the data-collector is working again and synced with the latest data. I will still investigate the problem and try to optimize the import to make it more "real time".

[rcavaliere]

@dulvui thanks for your effort. I see now that the data are updated to yesterday morning. To have still further issues in following the data stream?

[dulvui]

@rcavaliere The is still the problem that it takes around 24 hours to do the import and since now it had to import data of a whole week it took even more. Now it should update every ~24 hours but I'll check out the code to see how to make it more "real time" by optimizing the API calls.

[rcavaliere]

@dulvui clear... sorry for having asked!

[dulvui]

@rcavaliere The data collector takes now less than 20 minutes to make a full import :rocket: For now it is only in testing. I will see if everything works and put to production next week.

[dulvui]

@rcavaliere There is now a problem with the active stations. Currently on testing there are much less active stations than in production, because since now the data collector runs circa every 25 minutes instead of every 24 hours and only the signs that had data the last 25 minutes are set to active, all the other signs have active false. Let's check together in the office if this actually is a problem and in case I can increase the scan window.

[rcavaliere]

@dulvui yes it is a problem, we should avoid that a VMS that does not have an update is set to active = false. Can we decouple the fact that we have real-time data with the update of the status of the station?

[dulvui]

@rcavaliere Yes we can. The status change happens when a sync station is called and a station is no longer in the list, then it gets the field active = false. Currently the station sync and the data push happens in the same call every 25 minutes, so just a few stations are synced and all the others are active = false. I could modify it so that the data push remains the same every 25 minutes and the station sync happens only every 24 hours (?) or so. Would 24 hours be fine or should I put even more?

[rcavaliere]

@dulvui ok that's the right way to proceed. I would put 7 days at maximum, because it can happen that a VMS does not update for a few days if there is no events on the highway... The consequence of this is that we have active = true for longer period for VMS that are not active anymore, which is fine for me.

[dulvui]

@rcavaliere Now the fix is live in testing and there are around 570 active stations of a total of 685 stations. Before there where like 30 active stations. Currently in production there are 675 active stations of 685, but since in testing we only look for data in the last 7 days there are less active stations. On production we currently check for the past 3 years more or less. I will monitor the data until tomorrow and then put to production if everything looks fine.

[dulvui]

@rcavaliere Its now live in production and working fine