Installation seems to break built-in Transport NSW integration

[alexeiw123]

I have set up the built-in integration using the following yaml under sensor::

- platform: transport_nsw
  name: "morning_bus"
  stop_id: "<hidden>"
  route: "<hidden>"
  api_key: !secret TfNSW_API_Key

- platform: transport_nsw
  name: "afternoon_bus"
  stop_id: "<hidden>"
  route: "<hidden>"
  api_key: !secret TfNSW_API_Key

This works fine until I install the ha_transportnsw integration (using HACS) and configure in yaml. The yaml I'm using is:

- platform: ha_transportnsw
  name: "morning_bus_location"
  origin_id: <hidden>
  destination_id: <hidden>
  transport_type: 11 # Only school bus
  return_info: verbose
  api_key: !secret TfNSW_API_Key

- platform: ha_transportnsw
  name: "afternoon_bus_location"
  origin_id: <hidden>
  destination_id: <hidden>
  transport_type: 11 # Only school bus
  return_info: verbose
  api_key: !secret TfNSW_API_Key

From that point forward, I will get errors from both the built-in and this custom component.

This custom component error log:

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:727
integration: Sensor (documentation, issues)
First occurred: 9:26:29 AM (2 occurrences)
Last logged: 9:26:29 AM

ha_transportnsw: Error on device update!
Traceback (most recent call last):
  File "/config/custom_components/ha_transportnsw/sensor.py", line 287, in update
    ATTR_DEPARTURE_TIME: convert_date(_data["journeys"][self._index]["departure_time"]),
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ha_transportnsw/sensor.py", line 88, in convert_date
    temp_date = datetime.strptime(utc_string, '%Y-%m-%dT%H:%M:%SZ')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/_strptime.py", line 554, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/_strptime.py", line 333, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data 'n/a' does not match format '%Y-%m-%dT%H:%M:%SZ'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 727, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1302, in async_device_update
    await hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ha_transportnsw/sensor.py", line 230, in update
    self.data.update()
  File "/config/custom_components/ha_transportnsw/sensor.py", line 306, in update
    _LOGGER.error(message + " --- " + _data)
                  ~~~~~~~~~~~~~~~~~~^~~~~~~
TypeError: can only concatenate str (not "dict") to str

The built in error log is:

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:727
integration: Sensor (documentation, issues)
First occurred: 8:48:43 AM (2 occurrences)
Last logged: 8:48:43 AM

transport_nsw: Error on device update!
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 727, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1302, in async_device_update
    await hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/transport_nsw/sensor.py", line 128, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/components/transport_nsw/sensor.py", line 160, in update
    _data = self.tnsw.get_departures(
            ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransportNSW' object has no attribute 'get_departures'

I experienced this a few weeks ago and was unable to resolve - until I received a HA-core update; then all was working again (without this custom component added on). This time I took a backup and re configured this so that I could do location tracking and the errors have returned. In a real head scratcher to me - restoring the backup has not resolved the error!

[alexeiw123]

Also removing the custom component does not resolve the issue. Does this custom component modify files in the /usr/src/homeassistant/homeassistant/ directory?

[andystewart999]

Hi - I'll see if I can recreate this, but no, it doesn't modify any core Home Assistant files.

I specifically gave my integration a separate domain name so it shouldn't have any conflicts with the built-in one, but I'll see if I can work out what's going on.

[alexeiw123]

Yes it's had me baffled. This morning there was another home assistant core release (which is what fixed it last time). I didn't check the built in integration but removed the comments on this integration.

I still get the same error in my logs but one of my two school bus trackers got created and was updating OK. Maybe the afternoon bus entity will appear once it comes in to service this afternoon.

I'm kind of looking for a blend of features from both - I really only want to track the known school bus route, but I want geolocation so I can trigger location based alerts. departure/arrival is pretty mixed for me because the bus only has a few scheduled stops as you just hail the driver at any bus stop along the route.

[alexeiw123]

Could it be related to the dataset access in opendata?

I've tried generating a bunch of different API keys and trying to authorise the API key, but it never shows that it's been accessed. There's nothing too specific in the instructions on this and I've had data come through 'at times'. Should this be updating?

[andystewart999]

OK, this is interesting...

The built-in Transport_NSW integration uses the PyTransportNSW library, whose root class is TransportNSW. My integration uses the PyTransportNSWv2 library, which is obviously different, but it also exposes TransportNSW as the class. So I'm thinking that my library is conflicting with the other one.

Removing mine should then fix the issue with the built-in integration but you're still having issues, and I've managed to re-create that. I'll see if there's a quick fix to get the built-in integration working, but the better fix is for me to make my Python library unique and roll out an update to both the library and the integration, which I'll do ASAP.

[alexeiw123]

Removing mine should then fix the issue with the built-in integration but you're still having issues, and I've managed to re-create that.

This is what I find really hard to get my head around. I even took a backup when the built-in was working and restoring the full backup didn't resolve it until the next HA-core update.

[andystewart999]

OK, I've fixed the name conflicts and added some additional error checking while I was at it, so if you refresh HACS you should see version 1.0.2 is available.

Unfortunately this hasn't fixed the TransportNSW library issue, I'm going to raise a ticket with the Home Assistant guys to see if it's a known issue, and we'll also see if the problem goes away with the next minor or major release of HA itself.

Let me know how you go!

[alexeiw123]

OK, I've fixed the name conflicts and added some additional error checking while I was at it, so if you refresh HACS you should see version 1.0.2 is available.

Unfortunately this hasn't fixed the TransportNSW library issue, I'm going to raise a ticket with the Home Assistant guys to see if it's a known issue, and we'll also see if the problem goes away with the next minor or major release of HA itself.

Let me know how you go!

Awesome, nice one and thanks!

After an update I have entities being created, populated with N/A but I believe that's to be expected when I'm trying to track specific school buses.

I did create an issue in the core github: https://github.com/home-assistant/core/issues/123965 but I'm not sure that this captures the problem.

edit:

I see that the core integration error has changed on startup:

Logger: homeassistant.config
Source: config.py:1411
First occurred: 1:48:56 PM (2 occurrences)
Last logged: 1:48:56 PM

Platform error: sensor - cannot import name 'TransportNSW' from 'TransportNSW' (unknown location)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config.py", line 1411, in _async_load_and_validate_platform_integration
    platform = await p_integration.integration.async_get_platform(domain)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1100, in async_get_platform
    platforms = await self.async_get_platforms((platform_name,))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1195, in async_get_platforms
    platforms[platform_name] = await future
                               ^^^^^^^^^^^^
ImportError: cannot import name 'TransportNSW' from 'TransportNSW' (unknown location)

[alexeiw123]

@andystewart999 is this related to the changes you've made? Sensors are OK now and school buses are being detected but I've got no lat/long coming through, even thought bus is tracking in real-time (confirmed using anytrip app).

[andystewart999]

I'm still getting bus location info, where available. If you're happy to do so, you can privately send me your config and I'll test it specifically.

Also, if you've got a spare Linux machine knocking about you can use this Python script to see the raw output, just adjust the starts/stops as necessary. You'll need to install the TransportNSWv2 module of course.

from TransportNSWv2 import TransportNSWv2
tnsw = TransportNSWv2()

#Gordon station to St Ives
journey = tnsw.get_trip('207210', '207541', 'YourAPIKey', raw_output = True, transport_type = 11, strict_transport_type = False)
print (journey)

[alexeiw123]

If you're happy to do so, you can privately send me your config and I'll test it specifically.

Thanks - I sent you an email at the one in your profile. Seeing the same today, no location on services that have location in anytrip.

[andystewart999]

I've created new releases of PyTransportNSWv2 and this integration, fixing the conflicting Python module issue and also a number of other bugfixes. Thanks @alexeiw123 for helping with the testing!

[radman3362]

I'm still getting bus location info, where available. If you're happy to do so, you can privately send me your config and I'll test it specifically.

Also, if you've got a spare Linux machine knocking about you can use this Python script to see the raw output, just adjust the starts/stops as necessary. You'll need to install the TransportNSWv2 module of course.

from TransportNSWv2 import TransportNSWv2
tnsw = TransportNSWv2()

#Gordon station to St Ives
journey = tnsw.get_trip('207210', '207541', 'YourAPIKey', raw_output = True, transport_type = 11, strict_transport_type = False)
print (journey)

Hi Andy, Is there any specific steps to install the PyTransportNSWv2 module? I'm not sure if I have installed this module in HA.

I am having problems where the installation of the ha_transportnsw has broken other HACS integration. I have done testing over the last two days. Once I commented out the ha_transportnsw sensor config, all the other HACS integration came back. But if I have the ha_transportnsw sensor 'activated', some of the rest of the HACS integration will break.

Currently I am on the latest ha_transportnsw release (1.2.3). Thanks.

[andystewart999]

PyTransportNSWv2 is installed automatically by HA.

When you say that some HACS integrations break, can you be more specific? Perhaps include some screen shots or logs?

[radman3362]

Thanks Andy. I have spent some more time tonight, and confirmed that it is not breaking other HACS integration. However, there is an issue around the longitude/latitude returning n/a. I will open a new issue about this. thanks.