search

COVESA / vehicle_signal_specification

Vehicle Signal Specification - standardized way to describe automotive data
Mozilla Public License 2.0
326 stars 168 forks source link

Support of multiple measurement units in VSS signals #690

Closed nickrbb closed 9 months ago

nickrbb commented 1 year ago

In PR#669, a discussion was started on the issue of different units of measurement being used by different customers. To recap, current we define a "default" unit, which is mostly, although not always, in metric. However, there is a large proportion of the world that doesn't use metric (or whatever default is specified) and so the default unit will not suit them. A common example is that of vehicle speed, where seemingly one half of the world's countries use kilometres per hour and the other half use miles per hour. This results in unit conversion always having to be done at entities that requested the signal and received a response in a different unit to the one that they prefer/require e.g. kilometres per hour to miles per hour.

In a client-server model, this effectively means all clients having to implement code to do such conversion, whereas it would be more efficient to have the server do the conversion needed and save duplicate code amongst client applications. BlackBerry has also observed that some in the industry do not follow the COVESA specified default unit and instead use their own preferred unit instead. This requires client applications to "know" that the signal information returned in a response contains a unit of measure different to that defined in VSS.

Therefore, it is proposed to enhance VSS to accommodate different preferences (potentially on a per regional basis) of measurement units rather than instil a single unit of measure on the whole industry, as we effectively do today. To accomplish this, it is proposed to enhance signal definitions to include multiple measurement unit types as part of the VSS signal path name. This then enables VSS to accommodate for a plethora of different unit types, and potentially also simplify client pre-configured "knowledge" of systems and unit conversions.

As an example for this, consider the Vehicle.Speed signal. This could be extended to include all possible units specified in the defined speed "quantity" (previously called "domain") e.g. Vehicle.Speed.mph, Vehicle.Speed.kmh. Or to put it another way, we introduce branches to signals, providing one branch per possible measurement unit type as defined in the units.yaml file. This could even be automated via vss-tools e.g. every time vss-tools finds a measurement unit domain indication, it could replace it with branches associated with each measurement unit; Vehicle.Speed.[quantity:speed] -> Vehicle.Speed.kmh, Vehicle.Speed.mph.

This way, a VSS "client" can request the unit it prefers and the VSS "server" can take care of any needed conversion and provide the requested signal in the requested unit. If no unit is provided, then we could also allow the default unit (as we already specify) be used instead. This would also allow for some backwards compatibility.

Comments welcome.

doganulus commented 1 year ago

I think there is confusion regarding the role of the standard catalog here. Despite all the good hopes and intentions, the standard catalog cannot be used universally, as I try to explain here. Different systems will have different signal specifications, but hopefully, all standardized by the VSS.

So system developers will define the speed signal with their desired unit. The clients will understand which is which via vspec or schema files.

erikbosch commented 1 year ago

I see two possible paths here. One is to explicitly specify already in source .vspec file which units that are supported for a specific signals, the other is to rely on "dynamic unit support" based on for example unit files. If we go for the latter approach nothing needs to be changed in the source .vspec files themselves, it is up to the protocol or backend to provide a method to request the signal in different units, if the backend or protocol supports this. A typical example could be VISS/HTTPS, where it would be possible to use calls like:

Call Result
GET /Vehicle/Speed HTTP/1.1 Get speed in default unit (km/h)
GET /Vehicle/Speed/km/h HTTP/1.1 Get speed in km/h (we may need to discuss if we should allow / in unit identifiers, makes life more complex)
GET /Vehicle/Speed/Knots HTTP/1.1 Get speed in knots
GET /Vehicle/Speed/Mph HTTP/1.1 Get speed in miles per hour

Then one could of course discuss how a VISS-server would be able to support this. Is it it sufficient to get the *.vspec files as they look today and then a separate unit/quantity file, or would implementation be simplified if vss-tools could generate an "expanded VSS model" where all possible paths are present?

My view for now is that I have some doubts on if it is a good idea to explicitly specify multiple supported units for a signal. We could easily end up in long discussions on usable units for every individual signal. Like should we support mm and cm for Vehicle.TripMeterReading?, or only m and km. Isn't it better if an implementation either only supports the default unit (today km) or supports all units of quantity length?

nickrbb commented 1 year ago

Thanks @doganulus and @erikbosch for your comments.

@doganulus I agree that system developers can use implementers' vpsec files to tailor their applications for the target system. However, such tailoring takes time, money, and resources, so if we can simplify the needed tailoring, then surely that is a positive outcome for the industry?

@erikbosch The use of vss-tools to generate the expanded model in VSS definitions was certainly my preference. Regarding what multiple units would be needed, I agree and share your concern. We certainly don't want absolutely every possible measurement unit being provided on the off-chance that someone might need it in a road vehicle. To that end, I think we should limit the measurement units to those that have a clear and significant market need. Going back to the Vehicle.Speed example, there is a clear and significant market need for the measurement units of MPH and KM/H, with different regulatory authorities in different regions around the world mandating speed be provided to vehicle drivers in those units. For vehicle speed to be provided in knots, I know of no regulatory authority that requires this for a road vehicle (I am willing to be educated here though! :-) ). Similarly, is there a clear and significant market need for mm and cm to be used in trip meter readings? Again, limiting ourselves to clear and significant market needs would suggest no.

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units. I was not looking to expand VSS to provide every single possible measurement unit, as I agree that that would severely overly complicate implementations e.g. VISS servers. To that end, we could also perhaps agree to limit ourselves to a maximum of say 3 default measurement units per signal. Of course, some signals may still be OK with less e.g. a default of 1, based on market need, and that is fine.

doganulus commented 1 year ago

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units.

Hi @nickrbb, assume there is an integer on the speed signal. How should we interpret it? Unless specified otherwise, this value is interpreted using the default unit. If defaults are multiple, we still do not know how to interpret signals. So, they must be singular values by definition.

If different API endpoints are desired, then they must be defined as separate signals, with each having a default unit. This is fine for custom VSS server implementations, and for the standard catalog if the community agrees. However, it is less elegant and creates a precedent for numerous similar cases, which may be problematic in the long run. I have seen several projects going that way and getting bloated at the end, unfortunately. Therefore, it is better to avoid it, at least for the standard catalog.

A better solution to the issue is to standardize an overriding mechanism in the VSS. That amount of tailoring, I think, is unavoidable, even if I agree that less tailoring is much better.

nickrbb commented 12 months ago

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units.

Hi @nickrbb, assume there is an integer on the speed signal. How should we interpret it? Unless specified otherwise, this value is interpreted using the default unit. If defaults are multiple, we still do not know how to interpret signals. So, they must be singular values by definition.

If there is only one signal for speed (i.e. Vehicle.Speed), then yes, we need to specify a default unit, and hence we have the issue I describe above about what what measurement unit should be picked. Having multiple defaults for one signal should never be done, and just to be clear, I'm not suggesting that.

If different API endpoints are desired, then they must be defined as separate signals, with each having a default unit.

Agreed.

This is fine for custom VSS server implementations, and for the standard catalog if the community agrees. However, it is less elegant and creates a precedent for numerous similar cases, which may be problematic in the long run. I have seen several projects going that way and getting bloated at the end, unfortunately. Therefore, it is better to avoid it, at least for the standard catalog.

Well, that's kind of a philosophical debate, and I'd rather stick to the problem in hand here. Whilst I agree that bloat can be an issue in any project, I have also seen projects fail because they fail to take into account market needs.

A better solution to the issue is to standardize an overriding mechanism in the VSS. That amount of tailoring, I think, is unavoidable, even if I agree that less tailoring is much better.

The overriding mechanism in VSS is what we have today and is what is causing the issue. The proposal here is to more gracefully allow modifications to VSS by extending certain branches and therefore encouraging their use/non-use, rather than encouraging their overriding which is causing the issue I describe above.

Maybe I'll put some slides together for our call today, to clarify.

erikbosch commented 12 months ago

Added it as a topic for the meeting today , see https://wiki.covesa.global/display/WIK4/VSS+Meeting+Topics+and+Meeting+Notes

erikbosch commented 12 months ago

Meeting notes:

Nick presented the topic

doganulus commented 12 months ago

Thank you for the today's discussion. I think there is a valid use case for @nickrbb's case but I am unsure about expanding the signal tree (leaf nodes) by units, but the other ideas are more interesting.

I think the API fragment (#) solution is nice. The query syntax (?) is standardized in the URIs and we could write it such as /vehicle/speed#?format=mph. Moreover, this may handle accessing struct elements such as /a/vss/struct#/its/nested/property using the JSON pointer syntax. Overall, I think this idea could work and is open to expansion.

Besides the solution above, I think tailoring the catalog for different cases is unavoidable in general. Maybee manufacturers might want to prepare different vehicle.vspec files for Europe, North America, Japan markets etc.. This may be done for each vehicle model as well. But it's eventually up to the client to read vehicle.yaml and configure themselves for different cases automatically.

We should use VSS files for system and app configurations.

nickrbb commented 12 months ago

Support of multiple measurement units in VSS - 2023-11-21.pdf

Please find attached the slides I presented yesterday.

Just to reiterate, the problem is that of identifying the measurement unit of a received signal. It is not the issue of conversion, as some on the call may have misunderstood yesterday.

doganulus commented 12 months ago

the problem is that of identifying the measurement unit of a received signal

Could you please elaborate more on it with different words? In my understanding, we have a pub-sub system, and there is a topic name /vehicle/speed. We want to interpret raw numeric values on the topic and pick a default unit for the topic. Then:

What's the third way?

erikbosch commented 11 months ago

Meeting discussion:

Nick

Continue discussion next meeting (Dec 12th), to see what solutions that the community seems to prefer

erikbosch commented 11 months ago

Thought a bit more on this issue, and tried to structure my thoughts. Maybe we can use this as a decision structure, as I see different aspects of the discussion

Do we need to change VSS "core" syntax (the syntax used in *.vspec files in this repo)

Do we need to change the syntax or content of unit/quantity-files

Do we need to change what VSS says about APIs?

VSS is in itself not an API, but as part of VSS-project we do state some assumptions on APIs and by tooling we in some way set the standard, like:

With this said - there is nothing prevent implementations like Kuksa to offer an API where you could access for example Vehicle.Speed in Miles per hour by using Vehicle.Speed.Mph as identifier. But we could be more specific about this and add an "API recommendation" to the VSS documentation, maybe something like:

If an API use dot-notated VSS names and supports accessing a signal with alternative units, the identifier <VSS signal name>.<unit> (Like Vehicle.Speed.Mph) may be used by APIs to refer to the signal with the specified unit.

Example:

get Vehicle.Speed # Shall give current speed in the default unit used by the server, if using standard VSS km/h
get Vehicle.Speed.mph # Shall give current speed in miles per hour

It is optional for an API to support alternative units, but if it does it must make sure that the values are synced.

Then it is up to whoever wants alternative unit support to influence VISS, Kuksa and other APIs so that they support it.

Do we need to change vss-tooling

Performance and Implementation aspects

There were concerns that you might get duplicate topics (if using MQTT) when supporting alternative units. That could be the reality if clients are allowed to use any unit they want and they all interact directly with the topic and the MQTT topic API uses different topics for different units. That may be a problem in some implementations, but not necessarily in all.

Even if you use MQTT you could have an API where the dot-notation only is used in the programmatic API/SDK, the SDK may when getting the request below ask server for the actual unit for Vehicle.Speed and do conversion on client side and use Vehicle.Speed when communicating with the server.

subscribe("Vehicle.Speed.Mph",*MyCallback)

So the transport API does not not necessarily need to know or handle the <VSS signal name>.<unit> convention, it could exist on programmatic level only. So how it is handled in implementations could be quite flexible.

ppb2020 commented 10 months ago

@erikbosch, I like your approach, so I'll poach it to provide my feedback... ;-)

Do we need to change the syntax or content of unit/quantity-files

I agree we do not need to change the VSS syntax. All of this can be handled in the APIs, in whichever way the API author determines is best for that API.

Do we need to change the syntax or content of unit/quantity-files

On this topic, I must admit that I am challenged to understand the rationale behind what is currently being discussed outside of this issue regarding units, domains, factors, conversions, etc. Although I understand the concepts being shared and associated discussion (I understand domains, factors, etc.), I need to spend some more thinking time grokking the rationale behind the need to support domains, factors, conversions, etc. in the unit file. Thus, I'll hold comment on this part until I am more comfortable understanding rationale.

Do we need to change what VSS says about APIs?

In my mind, VSS should say very little, if any, about APIs. I can see it making statements like: "This approach or restriction was chosen in order to allow for more flexibility in how APIs are designed and implemented.".

The specification should generally avoid making recommendations about APIs, leaving as much latitude as possible to their implementers. Indeed, a separate project, such as Kuksa Val could certainly make certain recommendations on how to represent requests for signal values in a specific unit as well as what unit (scale, datatype, etc.) conversions it supports.

Do we need to change vss-tooling

I don't see the need to.

The "unit-extension" idea you have is interesting, but I do not see it as necessary based on my statement above that I'd leave all of this to the API implementation.

Performance and Implementation aspects

Again, I'd leave those concerns to the API implementation. Presumable, implementers would select the approach that best benefits their needs and achieves their performance goals.

erikbosch commented 10 months ago

Meeting notes: