mark1foley / ha-gtfs-rt-v2

Real-time transit information for Home Assistant
MIT License
45 stars 17 forks source link

Arrival times without a time field are ignored #11

Open calvinbui opened 1 year ago

calvinbui commented 1 year ago

Hey there đź‘‹

Your fork has been the most up to date I have found.

I have found that trains that arrive at my station without a time field in the StopTimeEvent type are ignored.

In the GTFS reference, it says that Either delay or time must be provided within a StopTimeEvent - both fields cannot be empty.

The code however only checks for the time field:

https://github.com/mark1foley/ha-gtfs-rt-v2/blob/ecca8d9ac9a08bca75534170468a8d49834fc14f/custom_components/gtfs_rt/sensor.py#L243-L251

For example, the entity in HA shows the next train in 16:48 and the one after at 19:50:

image

However, there's a train in 16:33 it's missing and many more between 16:48 and 19:50:

image

And looking at the API response, the train at 19:50 has a arrival.time value


    stop_time_update {
      arrival {
        delay: 0
        time: 1676019000
      }
      departure {
        delay: 0
        time: 1676019030
      }
      stop_id: "2137171"
    }

and the trains that do not have a lot more information, but do not have the time field, only a delay field:

    stop_time_update {
      arrival {
        delay: 0
      }
      departure {
        delay: 0
      }
      stop_id: "2137171"
      schedule_relationship: SCHEDULED
      departure_occupancy_status: MANY_SEATS_AVAILABLE
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 1
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 2
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 3
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 4
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 5
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 6
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 7
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
      [transit_realtime.carriage_seq_predictive_occupancy] {
        position_in_consist: 8
        departure_occupancy_status: MANY_SEATS_AVAILABLE
      }
    }
mark1foley commented 1 year ago

Hi,

Your absolutely correct in that if the StopTimeUpdate does not include an arrival time or departure time then it will ignore it.

The way the integration is currently written then it can’t do much else. The solution would be to lookup the GTFS static timetable using the trip_id and retrieving the scheduled arrival time from there (and then make any adjustments required using any delay specified). I have been thinking about integrating the with GTFS static data to retrieve rich data like the headsign (e.g. “Central”) and the route names (e.g. “Berowra and Hornsby to North Shore via City”) to populate the sensor with.

There is already a GTFS integration in Home Assistant core (https://www.home-assistant.io/integrations/gtfs/ ) that loads the GTFS zip file (under the GTFS Specification the format is a zip file) and extracts it into a SQLite database. However, as it is currently written, it requires the user manually download the file and place it in the gtfs folder. Obviously, this could be improved on significantly by having the integration download the file directly from the URL and automatically update the database on a schedule.

So, I think merging the GTFS core and the gtfs-rt integrations into one is probably the best option going forward. But I would need to talk to the HA developers about this.

I also need to modify the integration to support the operator’s gtfs-realtime.proto file, which is where they define their individual implementation of GTFS realtime. There is lots of variation in implementation with some of the new features like occupancy levels. This makes it painful when you try and develop an integration that is used everywhere.

What are your thoughts?

Mark

From: Calvin Bui @.> Sent: Friday, 10 February 2023 3:36 PM To: mark1foley/ha-gtfs-rt-v2 @.> Cc: Subscribed @.***> Subject: [mark1foley/ha-gtfs-rt-v2] Arrival times without a time field are ignored (Issue #11)

Hey there đź‘‹

Your fork has been the most up to date I have found.

I have found that trains that arrive at my station without a time field in the StopTimeEvent type are ignored.

In the GTFS referencehttps://gtfs.org/realtime/reference/#message-stoptimeevent, it says that Either delay or time must be provided within a StopTimeEvent - both fields cannot be empty.

The code however only checks for the time field:

https://github.com/mark1foley/ha-gtfs-rt-v2/blob/ecca8d9ac9a08bca75534170468a8d49834fc14f/custom_components/gtfs_rt/sensor.py#L243-L251

For example, the entity in HA shows the next train in 16:48 and the one after at 19:50:

[image]https://user-images.githubusercontent.com/3604363/218008539-5cc11206-7046-4156-b642-8bcc0c0482df.png

However, there's a train in 16:33 it's missing and many more between 16:48 and 19:50:

[image]https://user-images.githubusercontent.com/3604363/218008689-f4809eff-5dab-4bfe-aeb3-1617cc846a86.png

And looking at the API response, the train at 19:50 has a arrival.time value

stop_time_update {

  arrival {

    delay: 0

    time: 1676019000

  }

  departure {

    delay: 0

    time: 1676019030

  }

  stop_id: "2137171"

}

and the trains that do not have a lot more information, but do not have the time field, only a delay field:

stop_time_update {

  arrival {

    delay: 0

  }

  departure {

    delay: 0

  }

  stop_id: "2137171"

  schedule_relationship: SCHEDULED

  departure_occupancy_status: MANY_SEATS_AVAILABLE

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 1

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 2

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 3

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 4

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 5

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 6

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 7

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

  [transit_realtime.carriage_seq_predictive_occupancy] {

    position_in_consist: 8

    departure_occupancy_status: MANY_SEATS_AVAILABLE

  }

}

— Reply to this email directly, view it on GitHubhttps://github.com/mark1foley/ha-gtfs-rt-v2/issues/11, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL6CAIFND5QTC4OYJ6CCKSDWWXHVFANCNFSM6AAAAAAUXMQCZU. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

calvinbui commented 1 year ago

So, I think merging the GTFS core and the gtfs-rt integrations into one is probably the best option going forward. But I would need to talk to the HA developers about this.

The GTFS integration page mentions there is no realtime support due to protobuf parsing issues.

I also need to modify the integration to support the operator’s gtfs-realtime.proto file

The user should be providing this file if possible.

So, I think merging the GTFS core and the gtfs-rt integrations into one is probably the best option going forward. But I would need to talk to the HA developers about this.

Definitely.