Nextbus Integration existing Entities stopped working, deleted, and can't reload them.

android52 commented 2 months ago

The problem

After upgrading to 2024.8.0, I expected attributes to stop working as per the breaking changes, but it seems all my existing entities stopped working entirely. I removed the entities, and tried to re-add some (and others from the same transit system (Toronto Transit Commission), while they are successfully added (Sucess! message in UI), they are listed as "Needs attention" with the error "Failed setup, will retry: ('Failed updating nextbus data', NextBusHTTPError())"

I can succesfully add entities from other Transit agencies (i.e. APL, EZRide).

I'm not sure if this is an issue with Toronto (which has had issues in the past) or the integration.

What version of Home Assistant Core has the issue?

core-2024.8.0

What was the last working version of Home Assistant Core?

core-2024.7.? (believe I had updated to the last version)

What type of installation are you running?

Home Assistant Container

Integration causing the issue

NextBus predicitiosn

Link to integration documentation on our website

https://www.home-assistant.io/integrations/nextbus

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 2 months ago

Hey there @vividboarder, mind taking a look at this issue as it has been labeled with an integration (nextbus) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `nextbus` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign nextbus` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

nextbus documentation nextbus source _{^{(message by IssueLinks)}}

android52 commented 2 months ago

It seems either the 2024.8.1 update, a restart, or somethin in the ether resolved this issue since I opened it yesterday.

Today I am able to add routes/stops from the Toronto Transit Commission with no issues.

android52 commented 2 months ago

Seems I was hasty, It seems intermittent adding stops, even on the same route, some add and so do not.

ViViDboarder commented 2 months ago

What do you see in the log?

android52 commented 2 months ago

Unfortunately nothing that I can see.

The System logs show nothing (that I can see, perhaps I'm not looking in the right playce - System->Logs, nothing recent, then search for nextbus and there are no results). When adding a new entity (either one that works or one that fails), if I turn on Debug for the integration then create an entity then turn off the debug, the resulting downloaded log file shows nothing recent for nextbus. There is an earlier entry (from earlier tests where I had mulitple routes configured, but have since deleted these to try clean) with this failure success: False:

2024-08-13 13:36:00.959 DEBUG (MainThread) [homeassistant.components.nextbus.coordinator] Updating data from API. Routes: {RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx')} 2024-08-13 13:36:00.959 DEBUG (SyncWorker_8) [homeassistant.components.nextbus.coordinator] Updating data from API (executor) 2024-08-13 13:36:02.163 DEBUG (MainThread) [homeassistant.components.nextbus.coordinator] Finished fetching nextbus data in 1.204 seconds (success: False) 2024-08-13 13:36:07.401 DEBUG (MainThread) [homeassistant.components.nextbus.coordinator] Updating data from API. Routes: {RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx'), RouteStop(route_id='xxx', stop_id='xxx')} 2024-08-13 13:36:07.405 DEBUG (SyncWorker_45) [homeassistant.components.nextbus.coordinator] Updating data from API (executor)

Note that I replaced route_id and stop_id values with xxx for privacy, but can provide examples (which I tried to do when deleting everything and tried to recreate with other locations but got nothing in the log).

Please let me know what additional information could be useful.

ViViDboarder commented 2 months ago

I'm wondering if it's related to #123562. Let's see if 2024.08.3 helps.

android52 commented 2 months ago

I don't see 2024.8.3 available yet, but will watch for it. I am running 2024.8.2 still.

That works though today over mutliple entities. I had no problem adding 10 entities today from none (with the integration not even showing initially), where in the past the problem has happened adding to the existing integration or eveb going from no entities to a single entity. This isn't the same sort of error that is indicated in that other thread.

An example that was consistently not working before, but is working today was loading Toronto Transit Commission->Route 123 - Sherway-> Stop - 135 The West Mall (the first stop in the list). This consistently failed for me the other day but today is working.

I added that today and I kept going trying to add more stops.

TTC (Toronto Transit Commission) -> Route 10 Van Horne -> Stop 140 Pleasant View Dri.
TTC->11->2350 Bayview Ave - The Granite Club
TTC->324->2900 Warden Ave - Bridlewood mall

All seemed to work. I then went to try to add my specific stops of interest, all seemed to add that I was hoping for and could find (well I added 10, not all I wanted to at this point). However, various branches of specific routes do not show up

I ran into a few scenarios:

When the same route has multiple branches at multiple stops in the same station - sometimes there was an "AR" branch (assuming this is all routes), sometimes I could only find a single stop with one of the stop numbers.
When multiple branches service the same stops at a street corner
The "stop" number on the TTC's website doesn't match the stop number for the stop
The route/branch doesn't show up, but it is possibly because that branch isn't running at the time I am creating the entities

For one route, at the stop in the station it had listed an "ar" route which I think is "all routes", and specific branches but not all, or only one when there should be multiple.

Of the entities created today, many of the added stops/entities don't show the "Upcoming" Attribute, even though the same information on the Nextbus (i.e. https://retro.umoiq.com/webkit/#_predictions) and TTC websites seems to work.

At this point I've added 10 entities, some are different routes at the same Stop (Stop ID is the same), but not all branches of the routes at that stop show up. I've added multiple branches of the same route at different platforms (Stop IDs are different) in the same station (which is expected).

So, the original problem seems to be "gone" now (on 2024.8.2), but I'm still not seeing the results I'm expecting - primarily being no Upcoming attribute is showing, and secondary is the branch issues where not all branches are available.

Happy to discuss further and provide specific examples (though prefer not to do so publicly as this exposes my location)

android52 commented 1 month ago

I had a moment to upgrade to 2024.8.3 today. Before upgrading I noticed that many of the previously setup entities are failing even though they had been working when I wrote the above message. After the upgrade, things don't seem to have improved, and many entities are still failing (they try to initialize and then fail - 7 of the 10 I previously setup fail, 3 are ok over 2 different routes). The failure in the Integration screen is "Failed setup, will retry: ('Failed updating nextbus data', NexBusHTTPError()).

If I try to setup the same route/stop again I'm told the Service is already configured (expected). If I delete the entity and try to re-add it, it is stili telling me it is already configured.

What would you like me to test with 2024.8.3?

MReschenberg commented 1 month ago

I'm also having this problem (using the SFMTA provider in nextbus) -- if I delete/re-add the sensors they seem to work well up until I have ~4 routes or more. Then, they all go Unknown/Unavailable :(

ViViDboarder commented 1 month ago

I was able to reproduce after some time. It looks like there may be some kind of rate limiting or something, but occasionally a 404 is returned from the API.

I created a patch that may solve the problem. This both reduces the number of API requests as well as makes updates more resilient by only sharing a coordinator among stops rather than per agency. So if one stop returns an error, the rest of the entities may still update.

android52 commented 1 month ago

Thanks for continuing to look into this. Will watch for it to come through. I assume it will be in a future HA release?

ViViDboarder commented 1 month ago

Yea. The PR still needs a review though. So no target release yet.

android52 commented 1 month ago

Understood. Will keep an eye on the release notes.

ViViDboarder commented 1 month ago

Changes merged. Should be available in the next release.

android52 commented 1 month ago

Great news, thanks!

android52 commented 3 weeks ago

Thanks for your work - things are looking positive.

I upgraded to Home Assistant 2024.10.0 and removed and re-added the routes I was interested in tracking. I've added 15 entities (some routes have multiple branches) and so far haven't seen any issues with any of them failing to work.

Will have to let it run a bit, but this already looks better than what was happening before.

ViViDboarder commented 3 weeks ago

Glad to hear it! It’s still possible that with many sensors the API intermittently respond with errors, but this should prevent one error from breaking all sensors too.

home-assistant / core