Improve state and attributes of integration (re: Attributes like HVV)

[risiko79]

Hi akloeckner,

many thanks for your investigations taking the source from @kilimnik and put it into this repro. I'm searching for a DB integration for HA. I started with https://github.com/faserf/ha-deutschebahn but this implementation uses schiene too.

After pyhafas was fixed hopefully this project will get more boost.

Now my request:

It is possible to implement the atrributes of the sensor like the HVV integration? https://www.home-assistant.io/integrations/hvv_departures

This hopefully enables the possibility to use the hvv card https://github.com/nilstgmd/hvv-card

[akloeckner]

You mean, the following attributes should be filled with meaningfull data?

Attribute | Description -- | -- line | Line number of the next departure origin | The station where the transport started from direction | The station where the transport ends type | Type of the transportation, for example, Bus or S id | A unique identifier for the line. In most cases, line is sufficient to identify the line delay | Real-time data about the delay of the transport in seconds. Add this to the departure time to get the real departure time next | A list of the upcoming departures. Each element has the above attributes and departure containing the timestamp

I checked the HVV card, and it uses the next attribute to display the list of departures: https://github.com/nilstgmd/hvv-card/blob/5d73c08bd035957284a76fd80cc41288b3e0b75e/hvv-card.js#L81

However, the next attribute is already filled with just the single next departure. So, this cannot be changed to be a list, unless we break compatibility with the old db integration.

However, I also would like to have a list of departures in an attribute. So, maybe this can be added as a new attribute and HVV card can be enhanced to accept this new attribute, too? Or, we accept the breaking change and take this as an opportunity to re-think the entire attributes. In the latter case, I would also switch the state to be an actual timestamp...

[risiko79]

Hi. This is a new integration and from my point of view a breaking change should be acceptable. The old db integration is very simple and don't really support more departures. I come from FHEM and there the db integration is bretty much better.

Maybe HVV isn't the best example, but it supports more departures and a well card is available. From my point of view all departures/transport sensors in HA should be standardized (same attributes) like the weather sensors

MVG uses departures as list instead of next and you can define the number of departures with the attribute 'number' https://www.home-assistant.io/integrations/mvglive/

Do you know if @kilimnik will continue the development?

[kilimnik]

Hey, I am open to continue developing this further. The reason I used next, is for backwards comparability to the old db integration. But I think you're right, a list would be more usefull.

[kilimnik]

A standard transport system would be great. I'm not in contact with any HA Devs so I don't know if there is something planned.

[akloeckner]

Maybe we can have a specific look at other software (FHEM, HVV, HAFAS itself...) to see what would be a good format.

Also, it might be worth to start a discussion on the HA forums, such that other integration developers can find and join it more easily. If there are plans, someone on the forums should know. And if there aren't, we can propose something from scratch and discuss it.

I guess it will take a long time to agree on something, if that even ever happens. In the meantime, we can go forward with our own integration and find something that works for us.

[akloeckner]

I was waiting for my local transport company to become available in pyhafas, because I wouldn't have any meaningful use case without it. Now it is available and I have it working in HA. Shall we go on with this? As a start I created the forums topic: https://community.home-assistant.io/t/best-attributes-for-a-transport-integration/603036

[risiko79]

Many thanks for your investigations. Hopefully we will get answers asap. Maybe also interesting

https://github.com/agners/swiss-public-transport-card https://github.com/vas3k/home-assistant-berlin-transport https://github.com/MrGauz/home-assistant-munich-transport https://community.home-assistant.io/t/lovelace-munich-public-transport-departure-card/59622 http://hasl.sorlov.com/

A lot of individual implementations - sensors and cards

[akloeckner]

Thanks for these pointers. There is nothing in the community yet.

I'd summarize the integrations as follows:

• Some integrations make a difference between a station board and a transport sensor (swiss). The station board seems to be stripped version of the transport sensor.
• The state is usually the time of the next departure.
• The integrations all provide a departures or connections attribute with a list of next journeys.
• Some of the integrations provide data of the next departure as simple attributes.
• A journey can have the following fields (what a mess):
  • start, from, origin
  • destination, direction, to
  • train_number, line_name, color, linename, line
  • line_type, product, category
  • platform
  • departure, time
  • duration
  • remaining_time
  • transfers
  • trips
  • delay, canceled, prognosis
  • price
  • attribution

The pyhafas module returns data in the Friendly Public Transport Format. It seems a bit proprietary to the group around the module. We retrieve a list of Journey objects, which are in turn basically a list of Leg objects with the following fields:

• id (str) – ID of the Leg
• origin (Station) – FPTF Station object of the origin station
• destination (Station) – FPTF Station object of the destination station
• departure (datetime.datetime) – Planned Date and Time of the departure
• arrival (datetime.datetime) – Planned Date and Time of the arrival
• mode (Mode) – Type of transport vehicle - Must be a part of the FPTF Mode enum. Defaults to Mode.TRAIN
• name (Optional[str]) – Name of the trip (e.g. ICE 123) (maybe None)
• cancelled (bool) – Whether the trip is completely cancelled (not only some stops)
• distance (Optional[int]) – Distance of the walk trip in metres. Only set if mode is Mode.WALKING otherwise None
• departureDelay (Optional[datetime.timedelta]) – Delay at the departure station (maybe None)
• departurePlatform (Optional[str]) – Real-time platform at the departure station (maybe None)
• arrivalDelay (Optional[datetime.timedelta]) – Delay at the arrival station (maybe None)
• arrivalPlatform (Optional[str]) – Real-time platform at the arrival station (maybe None)
• stopovers (Optional[List[Stopover]]) – List of FPTF Stopover objects (maybe None)
• remarks (List[Remark]) – (optional) List of remarks

My thoughts so far:

• state should be the live (including delays) datetime of the next non-cancelled journey.
  • A datetime, because it can be unambiguously used in further processing. It can also be shown in the Lovelace UI as relative time ("in two minutes").
  • With delays, because that is what I am interested in, when going to the station.
  • Non-cancelled, because I noticed that DB also returns canceled departures, which then might show up as next departure.
  • The next journey (and not just departure), because we are interested in a possible complete connection, not just the next departure ending on a cancelled train later on.
• The attributes should simply reproduce the returned Journeys object. That's just what HaFaS returns.
• For convenience, the attributes could reproduce (possibly selected) data from the journey shown as state (departure data from the first leg, arrival data from the last leg).
• We could add an attribution to let the user know, that data is retrieved via HaFaS from xyz provider.

What do you think? (Note that this will not be compatible with the HVV card.)

[akloeckner]

The attributes should simply reproduce the returned Journey object. That's just what HaFaS returns.

I just tried that... it's not so easy, because the object seems to be recursing indefinitely. So, we'll have to make a choice which variables to return...

# see https://gist.github.com/sungitly/3f75cb297572dace2937?permalink_comment_id=4211196#gistcomment-4211196
def to_dict(item):
    match item:
        case dict():
            return {k: to_dict(v) for k, v in item.items()}
        case list() | tuple():
            return [to_dict(x) for x in item]
        case object(__dict__=_):
            return {k: to_dict(v) for k, v in vars(item).items()}
        case _:
            return item

[risiko79]

Hi. Sorry I can't support so much. But I think such a generic solution will not work. You have to go through the results from pyhafas and have to create a own dict by yourself depending on the journey or object type.

In your proposed return I can't see a list for the journeys. An attribute should return a list of dict (the converted journey object)

[sc-ampersand-p]

• Non-cancelled, because I noticed that DB also returns canceled departures, which then might show up as next departure.

I'd advise against filtering out cancelled connections. Especially when it comes to hourly connections, you might not even notice that the next connection you're looking at is an hour later than you expected it to be. Just my two cents, obviously, but I like the DB app for precisely that reason. In my area, the VVS app just removes connections from the search results instead of marking them as "they used to exist, but not today".

Obviously, this would mean somehow clearly marking cancelled connections. Not sure if that's properly feasible. Returning a connection that isn't even going to run without any indication that it's been cancelled is an obviously bad choice.

[akloeckner]

you might not even notice that the next connection you're looking at is an hour later than you expected it to be

Good point. I guess it also depends on the type of state. I'd opt for an actual datetime. It's unambiguous. It can easily be used programmatically. It shows in the front-end as relative time. So, this one would be obviously reporting "in 1 hour". If we were to stick with the legacy time format "hh:mm + d", that would be less obvious.

In the list of departures, we should keep the cancelled journeys, if the transport company provides them. They will be marked as cancelled, of course.

PS.

Returning a connection that isn't even going to run without any indication that it's been cancelled is an obviously bad choice.

That is what's happening right now with data from DB. There is an attribute canceled: true. But that is not visible directly in the front-end.

[akloeckner]

I'll close this via #7. Thanks for all your constructive feedback!