[FEATURE]: Having more realtime listed (more than next departure) for route

[hal01]

Me again... to bring additional features, at least for my usage of looking what are next buses coming on bus stop In results of a route sensor there is next departure time and in addition some departures following are provided in 'next_departures' attribute

With real time source, the next departure has the real time update in 'next_departure_realtime' attribute. Would it be possible to have the update also for the other departures (if available). ?

[vingerha]

Couple of challenges here, based on code and data experiences

1. realtime information is provided via route_id/trip_id and then the various stop_id without route/trip. The few ones that I use have nothing more then the next trip. This is often logical as why would you supply realtime data for trips that have not started yet...but this may be different in large(r) cities with multiple trips in minutes from one another. In the latest release there is an option to extract the realtime data in readable format (via service call), do check if it has more than 1 listing for your route/stop/direction
2. the current code uses the trip_id of the next departure (static data) and only that, in the attribs you see a list of multiple static-data depatures but there is no trip related. So, In order to do so I need to create a new sensor specifically for this, but 1. is stopping me from this, a lot of work with a high risk of not having real-time matching data...so let me know of your findings please

[hal01]

My usage is in fact the following : close to my house I have a stop Point with 2 lines of bus. Both of lines allow me or my children to go to college or library or shopping... I have a tab on the wall in kitchen with HA dashboard

I would like to show the next departures on table format, with real time (at peak time some delay .... ) to be able to have a quick look and decide which bus to take. So in how many minutes I have to be ready and leave house (less than 2 min walk to go at stop). Especially in winter when it's freezing it's interesting

I have also the usage of going to work every moring through a more regular line and route.

Today we are doing it through the app on the phone, but I would have find easier to do it like this. I am also digging for this feature on using the data given directly by the API of Ilevia (not in GTFS format) but using Node red to create a sensor which is containing all data with real time on a stop.

So don't spend hours of dev to this feature if it's only for me. Your integration is already great and useful for many other usage, including the other usage that I have. My final dashboard will certainly consist in a mix of your integration and API listing

Thanks again for your dedicated work

[vingerha]

So you are saying that your realtime data has multiple entries for the stop/bus you aim for? Again...where I live the frequency is 30 mins and I only receive updates for my stop about 5-10 mins before the bus arrives....if I receive them at all, Zou06 is not very reliable as compared to palmbus in Cannes. What I can try to do is to try and extract route-based info without going into the trips....can you attach the readable file here and point me to the multiple entries you'd like to see covered?

[vingerha]

Was just about to check this and quelle c.nn.r.. , zou06 removed the realtime data from their offering.

[vingerha]

What do you think of this?

[hal01]

Seems to be perfect for the usage I described !!

Thanks a lot

[hal01]

By the way, Any possibility to add realtime Departures to the "next departure around " option

I mean have real time as an information in the next departures around me as now it seems to be only the timetable (so times theorical) Adding here a Real time departure....

I imagine it's not so simple .... but just asking

I can Add a [new feature] request, but before , just asking if you think it's feasible ... before

[vingerha]

I have a solution but it is ugly.

The data comes from two different sources, one static, one realtime The one from static updates every (say) 10 mins and collects the various trips for the stops in time-range and lat/on radius, a low update frequency is fine as it is static. Per stop, there can be more routes and per route one or more trips, i.e. in cities you will find more due to higher frequency

Realtime / stop only works via trip_id, i.e. you want the rt data for that specific trip (bus/tram)

Since the code collects all routes/trips per stop in ONE sensor, I need to query both static and realtime each minute to also get the rt data, which is ugly. Wiht my limited knowledge, I blieve that cannot update sensor partially, i.e. have the realtime run per minute and the static slower.

You can test out what I have if you try this https://github.com/vingerha/gtfs2/releases/tag/0.4.0.1

[hal01]

Hello J'ai essayé de mettre le dossier téléchargé dans custom components (tout ce qu'il y a dans custom component de la pre-released) j'ai effacé le contenu du dossier _pycache Puis j'ai redémarré Home assistant

et je n'ai plus rien qui fonctionne. C'est bien cela la procedure d'installation ?...

Merci

[vingerha]

In HACS you just install GTFS2 and in the process you can select to have dev/beta versions

[hal01]

Yes done through Beta channel

So I tried and for the moment doesn't work... Entities & service are not created anymore from the same location... when I select "around position " function Before I used to have 1 service and 3 entites created

But Ok for th sensor created when creating a route ...

[vingerha]

Exactly my point above...it is quite a trick to keep all options in place. Where it worked for me, it crashed with your stops, impossible to cover all situation this easily :) 0.4.0.2 should help, I tested with the lat/lon you provided Note: I am still nto sure if this is the way to go, the code calls way too often to both sources and it now becoming very complex to understand.... will give it more thought

[hal01]

With 0.4.0.2 : working back, but Sensor is unstable (sometimes unavailable) when available, no Realtime shown and/or duplicate trips... Seems in fact tricky ...So maybe it's not a good idea.

On my opinion, the Realtime is maybe more important than theorical time..

I'm far from development and technical constraints on this ... but as a way to how it could be sort of double process in sensor updating 1/ First process : to build sensor data and trips with a low frequency and based on the time to look in the future define in configuration and distance from the geo point. 2/ setup a frequency of update in the same windows of real time url integration, the frequency will touch and update only realtime hours of the trips collected in the sensor, if it find any....

This is maybe what is already setup...

Thanks a lot for your work anyway !

Edit : Remark : on the 3 sensors created by the service around the location 2 have real-time data and 1 is not having (the direction of the 2 that having the RT data is the same and no data is the one in reverse) In all the cases route id are duplicated.

[vingerha]

I am repeating myself a bit, the idea of these sensors is to have 1 entity per stop_id which then can have multiple buses/trams/etc. Hence it has multiple routes and routes have multiple trips. ... as you alread noticed. I cannot query the attributes of a single trip with a high frequent real-time update, i.e. I need to page through each of the trip_id and then match it with any realtime data, each and every time the realtime search runs as in the meantime the content (static data) may have changed. Not impossible but work :) ... I do not want to split out routes in different entities as the stop-all-transport idea loses its value and I already know situation where one would have 5+ routes on a stop making it hard to deal with plus...it comes rather close to the other setup where you define based on a route and stop-origin and sopt-end

[hal01]

First just to illustrate the situation at the stop ... the difference between normal time and RealTime (and this at the beginning of the line and not peak hours)

So that's why I'm saying that I prefer have Real time than timetable hours at the stop for my usage.

But I think I have understood your problematics... So, not so simple, and thanks a lot for your time and involvement in this integration... I really appreciate your job and reactivity, so don't take bad any of my message , just to be sure, I wanted to clarify. As sometimes my english seems a little bit strict.

To come back on version 0.4.0.2, and additional tests that I did

I put french menu as I have french version 1/ For "Departs prochains autour" I have duplicated lines as shown above (2 lines for same route ID) Don't know if it's normal behaviour and I should adapt ow to present results or if it's due to the system complexity that you described

2/ for both "departs prochains autours" and "horaires pour stops" I don't have any real time when the sens is "retour". I mean

• I have real time data for Route define in the way "ALLER" when creating a stop on a route but not in case of a stop define in a " Retour" routes
• I have real time data for "Departs prochains autour" that are in the sens corresponding to sens "aller" but not in "retour" (exemple screenshot below - HeuresRT mean hours of Real Time)

I'm trying to find and help as much as I can

[vingerha]

Don't get me wrong ... I do understand and I also have seen the odd diff. between earlier realtime than the static ones. Even in Cannes with the Gare as first departure/stop sometimes minutes earlier, no clue but I can only use data=data. Maybe I should let go of combining static/realtime and create a separte realtime entity for the same stop, it is by far as nice but at least the two will not bug one another

[vingerha]

Added question, did you check if the real-time values actually match with 'reality"? It is nice to send data but if the bus/tram mismatches then what is the value

[hal01]

Yes all real time data (when present) are matching the Ilevia System and app So it's coherent of the real time information you have on bus stops digital boards , or app

and usually coherent with real life

[hal01]

Don't get me wrong ... I do understand and I also have seen the odd diff. between earlier realtime than the static ones. Even in Cannes with the Gare as first departure/stop sometimes minutes earlier, no clue but I can only use data=data. Maybe I should let go of combining static/realtime and create a separte realtime entity for the same stop, it is by far as nice but at least the two will not bug one another

Yes maybe a good opportunity to have real time schedule for the coming departure around .... and keep timetable "theorical" hours for an other usage....

[vingerha]

I will try to give it one more go for a combined solution towards the end of the week. Btw...the one I have setup with your lat/lon, 5 stops, seems stable with me...I of course do not look at it all the time :)

[hal01]

I will try to give it one more go for a combined solution towards the end of the week.

Ok let me know ifyou need some testing

Btw...the one I have setup with your lat/lon, 5 stops, seems stable with me...I of course do not look at it all the time :)

You hav real time in both direction ?? Even on stops that have an ID finishing by 1 (HLO001 for exemple) because yes I have fairly regular data , but missing Real time for "retour" directions on trips. see my sensor for this direction :

[vingerha]

I have no access now as travelling but I remember seeing the same, the '-' is added if there is no match for the trip and dire,ction. For me, it seems to work and I already know that the data is to be mistrusted too often so I am not (!) going to analyse the data for each provider.... this is undo-able. But you can do this yourselves via the service call, create a readable file and verify if you indeed can not find a RT entry for that route / trip and direction. If you have a proven bug, then I will respond :)

[hal01]

Hello I confirm that the data is present in the RT file or the route id for exemple route 4954173

and in the RT file extracted through the service

The timestamp in RT file for the stop and the route is coherent with real time data from Ilevia website & different from the theroical schedule time in the static file.

Working for stop HLO002 but not for HLO001

Addition :

Both stops are in stops.txt from static zip HLO001,Hautes Loges,AVENUE MARÉCHAL DE LATTRE DE TASSIGNY 59700 MARCQ EN BAROEUL - AU DROIT DU N° 265,50.66887,3.088711,1,,0,1148,,0 HLO002,Hautes Loges,AVENUE MARÉCHAL DE LATTRE DE TASSIGNY 59700 MARCQ EN BAROEUL - AU DROIT DU N° 258,50.668548,3.088579,1,,0,1148,,0

[vingerha]

It is not showing as the RT has no direciton_id and my query says trips_id and direciton id have to bmatch ...sometimes it has and sometimes there is a direction_id. Tbh, for trip you donot need a direction_id but I noticed cases in Palmbus where it does...sigh, BIG sigh

[vingerha]

This was an easy thing to change, try 0.4.0.3 please

[hal01]

This was an easy thing to change, try 0.4.0.3 please

Same result as for 0.4.0.2 ==> no real time for both sensors created by "departs prochains autours" and "horaires pour stops" for the direction ID =1 Ok for sensor created that are corresponding to route with Direction id = 0, there is real time data.

I opened the trips.txt from static file and there is direction ID in the trips (0 or 1)

But as you noticed no direction_id in realtime extraction

So it works (and was ok also before this version) when direction id =0 but not when direction_id=1

I checked and Data for trips are all in RT file extracted.

Thanks a lot for your help

[vingerha]

Pitty but not really a surprise as I can only do this remote without testing. After having had a next look I will have to reqrite this. Another reason is the massinve amounts of url calls, for each stop multiple trips and for each trip it will call the external rt url. With you that is at least 10 calls every minute and I have a local one in Basel which will reach 25+...for the SAME set of data....that is not good at all....need to rethink this too

[hal01]

Ok Thanks for your reply.

Would you like me to create issues on Github related to the 3 issues that I have currently for being more convinient ? because there is issues that are from beta version and some from normal version

For version 0.4 (standard) Duplicate value in sensor for "stops around" feature : each informatiopn of trip that stop at stop is doubled in sensor.

the 2 issues are for Experemental version 0.4.0.3 1/ Realtime for Route not giving result when direction Id is different than 0 and direction id is not in real time information from provider 2/ experimental feature of RT for stop around : when direction id of the stop and associated trips is different from 0 no RT information (+ the nomber of url calls that you describe)

Let me know what you prefere and I'm available for any help that I can provide with my low knowledge

[vingerha]

For now, leave it in this chain as they relate and I donot want to create more bug-records on stuff that I already know is not working ...and might never work :)

[vingerha]

Hi, 0.4.0.4 now seems to work, it is still by far from what I want as it calls RT to often. However, you can test if the data is at least correct. I am not sure yet how to solve th enumer of calls...posisbly I will download the rt data with the each first refresh call and use that locally.

[hal01]

Hello 👍 Great, working a lot better for 2 directions !!!! Direction 0 and direction 1 (so for all my sensors)

Both for route and stop near me... options of creating entities.

Yes could be interesting to fix a sort of frequency of downloading RealTime for all sensors or route that are attached to a source, that all sensor can use without calling the service

Maybe, it's certainly a lot of changes, but why not setting up Static and Real time sources on the same moment at a creation of a source. Instead of creating a source for static zip and after configuring on each service/entity the real time information. And maybe define a frequency of refresh for the source instead of a frequency by service ... ? Don't know, just exchanging

Thanks again, will continue to test

[hal01]

Sorry, In fact I found some kind of bug ... and why some departure where duplicated It can be solved for my usage by doing a filter on the realtime, ... so no emergency

GTFS format seems in fact very complidated ...

In fact seems that there is service days If in trips.txt I match the trip_id with the one in sensor for exemple 4960353

and with service_id I go the calendar_dates.txt (47854-__L559-S) I found this service_id for all dates that are linked to a saturday so March 23, 30 .... but indicated in the sensor on the 22

So as hours I changing a bit between saturday and days of weeks or sometime the same, taht why there is more values than what is in real time

To summarize , there is 12;20 bus on this line on Saturday 23rd but not today, today the bus is at 12:16 (and we can see a real time attached)

Thanks for adding real time it allow to discover this

[vingerha]

Honestly it is indeed a pita, the data is either managed through calendars or calendar_dates so I have to riun two queries againstthe data set and they of course have different output which needs to be combined later on to one view. Anyhow....try 0406 stp

[vingerha]

Maybe the most irritating part is that the delivery looks like the stuff I get from my developers....fix one thing, break aonother or find a new one. Major challenge remains the variety of data/providers and no time to test everything

[vingerha]

Still not good, I have another test that proves duplicates...although for you it seems to work...so will chnage code somewhere soon...

[vingerha]

Ah, no... I am seeing multiple stops for the same bus so that explains the almost-duplicate timings...still good.... I hope your tests will confirm that :)

[hal01]

On my side it seems OK now, No more multiple stops,

service and sensor more stable (before was erraticly unavailable sensor) Update to 0.4.06 at the line

[vingerha]

OK... I never looked at a sensor this way. I cannot easily understand why the static part of the sensor becomes erroneous but maybe this was related to crashing realtime.... Closing this ticket now :)

[hal01]

Great job thanks a lot for this improvement

[hal01]

And if you want to test improvement as discussed aboved, you can count on me.

[vingerha]

Sorry, got lost in all the back/forth...what improvement are you referring to?

[hal01]

If you would like to change the way Real time data are used etc ... as your were saying above ...

Edit : in fact it's seems that is part of change log already on version 0.4.0.5... but I switched directly from 0.4.0.4 to 0.4.0.6... so I didn't noticed, ...

Great , thanks a lot

Or you need tests to be done on further dev...

[vingerha]

Yep, I did a lot in 05. The most important change is that the rt data is always stored in www/gtfs2 for the local_stops. And althouigh the file is being read many times, it beats collecting the same amount via the internet. As I mentioned, in Basel I have a spot where 3 trams and 5 buses arrive every 10 mins, this means that for the next (say) 20mins) I have 35 calls/min for realtime, from a local file this is much less heavy Thanks for the detailed testing and helping feedback :)