'utf-8' codec can't decode byte 0xa2 in position 99: invalid start byte | when adding NYC subway real-time data source

[mbonaccors]

Describe the bug When I try to add real-time gtfs feed from the mta developer page as a new data source it is not able to parse the feed and create the database.

when I look at the files in the gtfs home assistant folder I see the following inside the

/homeassistant/gtfs2/Subway - ACE.sqlite

file:

'utf-8' codec can't decode byte 0xa2 in position 99: invalid start byte

url to data source (NYC MTA Gtfs real-time feed for A,C,E subway lines)

https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-ace

Release used

• gtfs2 release (latest version)
• HA type (HAOS/Container) (container on Debian)

Additional Please add logs if helpfull, switch to 'debug' logging to get more details

[mbonaccors]

All the NYC subway data feeds can be found here:

https://api.mta.info/#/subwayRealTimeFeeds

[vingerha]

Two issues:

• the integration expects a file to download, this I could work on ... but ....
• the content does not seem to adhere to protobuf, which I cannot fix

[mbonaccors]

Ah I see. Thanks for reviewing this so quickly! I’ve been trying to find an integration for home assistant that can connect to these subway feeds (many in NYC probably in same boat). Is there any more information I might be able to provide or be of assistance or is it basically not going to be possible with this current codebase?

[vingerha]

Hmm... for some reason it works for me (now).... so the format is correct (enough) for my lib to convert it. What I did was set it up via th service call, just the url and to check, I ticked the debug output toggle....it presents the converted json nicely. So it seems not the source itself. I need precise steps on how you set it up until it crashed.

[mbonaccors]

Hello again - I can try setting it up via the service call as well, is there documentation on that installation process?

Here are the precise steps on how I set it up from my end with screenshots:

1. HA info:

2. GTFS2 installation info:

3. Go to settings --> integrations --> add GTFS 2:

4. Obtain URL for ACE line of subway from MTA Developer resource page (Realtime Feeds):

5. Setup a new data source:

6. Wait a few minutes, review files in GTFS2 directory:

Seems not to be working, but let's confirm:

Try to create new "Scheduled Departure on Stops in your vicinity"

I looked in the "HA Logs" but do not see anything when i search for GTFS or Subway.

[vingerha]

You are using a realtime feed where you should use the static (!) data feed.... this is in (imo) clearly mentioned in the docs

[mbonaccors]

Ah okay, I understand, so there is just no ability to use realtime feed at the moment. I will use static feed for now. Thank you for reviewing once again.

[vingerha]

The realtime data is not giving a lot of information, it only has some (usually cryptic) pointers to route, trip or stop but no details what these would be

[mbonaccors]

Ah that makes sense upon reviewing the documents:

https://new.mta.info/document/134521 Seems they have a pdf that explains how to decode the information. Seems like a larger project. If it is feasible, might I add it as a future feature request?

[vingerha]

Highly unlikely as it states "extensions added specifically for NYCT", this integration only covers gtfs generically ... so you need to get the static data, for which I am almost certain that this exists.

[mbonaccors]

Yes, I now have it working with static information!

[mbonaccors]

Got it working with static. Many thanks!!