Filtering essential and optional

[tguild]

On a breakout call on VISS v1 experiences and v2 features, some of the filtering capabilities in v2 are considered potentially more complicated, less desirable to implement. As we want consistency in VISS implementations we should indicate which are required and which optional.

https://www.w3.org/2021/04/27-auto-minutes

In arguing for or against subscription filters, please indicate rationale to include or exclude. Do feel free to propose additional ones besides what is currently defined.

https://w3c.github.io/automotive/spec/VISSv2_Core.html#filter-request

[SebastianSchildt]

I do not want to argue for or against , but rather optionality

When implementing VISS "lower" in a software stack to facilitate access to VSS Signals, you may not need much more than an "abstraced field bus" or "MQTT-like" way to access data. You want the (quite central) software abstracting and providing the signals in VSS terms lean (for footprint, speed and code quality reasons).

If your use case involves more complex edge processing, probably some Edge stacks or components (such as IoTEA, DAPR, REDIS, Influx etc.) will be used, that provide these functionalities - probably even more powerful, on a more generic level. You would not want to replicate the functionality on lower levels.

This argument follows my mindset of using VISS inside a vehicle as access to the "in-vehicle digital twin". When imagining "VISS" used for a powerful all-in-one webservice to access vehicle data in a cloud backend, you might end up with different decisions.

As such I would argue a Base Profile VISS does not need to support any filtering. Of course VISS should still define filter capabilities deemed useful by the community, and maybe introduce a specific error message for "filter method not supported". That would also give way for future extensions or custom filters without breaking the protocol.

[peterMelco]

The conclusion that filtering should be optional I fully support. This is what is going to happen with implementations in any case, whether we specify it as mandatory or not. The considerations such as lean, speed and simplicicty will outweigh spec violations.

On Tue, 4 May 2021 at 06:31, Sebastian Schildt @.***> wrote:

I do not want to argue for or against , but rather optionality

When implementing VISS "lower" in a software stack to facilitate access to VSS Signals, you may not need much more than an "abstraced field bus" or "MQTT-like" way to access data. You want the (quite central) software abstracting and providing the signals in VSS terms lean (for footprint, speed and code quality reasons).

If your use case involves more complex edge processing, probably some Edge stacks or components (such as IoTEA, DAPR, REDIS, Influx etc.) will be used, that provide these functionalities - probably even more powerful, on a more generic level. You would not want to replicate the functionality on lower levels.

This argument follows my mindset of using VISS inside a vehicle as access to the "in-vehicle digital twin". When imagining "VISS" used for a powerful all-in-one webservice to access vehicle data in a cloud backend, you might end up with different decisions.

As such I would argue a Base Profile VISS does not need to support any filtering. Of course VISS should still define filter capabilities deemed useful by the community, and maybe introduce a specific error message for "filter method not supported". That would also give way for future extensions or custom filters without breaking the protocol.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/w3c/automotive/issues/381#issuecomment-831943375, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADAF77WADZYTKLDLGQSOSBLTL7ZKXANCNFSM43UYLEQQ .

[tguild]

@UlfBj and I discussed yesterday after attending Spatial Data on the Web call for Geofencing topic.I realized there could be a wide number of possible parameters including on/off work hour times, geofences, etc that might prompt different data collection. This issue is about having required and optional filters. Besides the filters we in the spec, I could see extensions that define additional. Client application should be able to query and get back various supported subscription types

At the very least we need to clarify which are essential, required filters to expect in every implementation. What's your list?

[UlfBj]

What the spec offers is the following:

Paths Filter: Multiple paths in one request History Filter: Request for data that has been saved by the server for some reason (e. g. temporarily out of coverage) Capture Filter: Time-based: Data returned on interval base Range: Data is returned when its value is within set range boundary(ies) Change: Data returned when change compared to previous value exceeds set limit Curve-logging: Data is processed by curve-logging algo before returned Metadata Filter: Static: VSS tree metadata is returned Dynamic: Other, "non-VSS", metadata is returned

Paths filter can be combined with History or Capture filters Range can include one or two boundaries. Capture filter can only be used for Subscribe requests. History and Metadata can only be used in Get requests

[UlfBj]

My view is that Paths and Static metadata filters shall be mandatory, all others can be optional.

[tguild]

Discussed on WG call https://www.w3.org/2021/06/15-auto-minutes.html Ted suggests time and change as mandatory since they were both part of VISSv1 Gunnar felt history and curve should be on optional list

@SebastianSchildt and @erikbosch - what are on your thoughts for optional/required lists? Earlier Sebastian argued that although filters are useful none should be mandatory but then without a filter what does one get or expect? Could be rather extensive depending on how VISS is implemented and integrated into the vehicle, service could try to send all translated CAN messages available as they come across on the bus..Alternatively one could argue sending no signals unless a narrower set is explicitly requested to avoid undo load.

[erikbosch]

Even if an VISS needs to support certain filter types to be VISS compatible, the use of filters are still optional in e.g. subscribeRequest. I do not know if we have specified what should be the behavior if no filter is given, or if that is implementation dependent. My assumption is that each VISS implementation will have a default behavior for handling subscriptions. This could be event-driven, e.g. react whenever a new value is received over CAN, and/or timer-based, e.g. check the value every second and send it out if it has been changed. I also believe that most VISS implementations will have implicit filters/limitations that are applied in addition to any filters given. For example, even if a change-filter is given, there might be an implicit period-filter that limits the frequency to e.g. at most once per second. For some scenarios the default subscription behavior might be sufficient, i.e. no need for filters.

In the transport section we state:

The server MAY reduce the number of subcriptionNotification messages sent to the client in order to reduce processing demands.

... so there is no promise that the server will fulfil the filter request. So even if the server e.g. receives a CAN value every 50 ms, it will unlikely send it out on the websocket every 50 ms. Have it been discussed how the server shall respond if it consider the filter to be unacceptable, e.g. requesting a too short timer interval or too many signals, so that the server can predict that quality of service will be bad. For example, reply with filter_invalid or not_acceptable?

By the way - what is the unit for period in time-based filters? Seconds? Milliseconds? It it specified somewhere?

time-based: the op-extra contains the period time X in between captures, {"period":"X"}.

 {
                "action": "subscribe",
                "path": "Vehicle/Drivetrain/FuelSystem/Level",
                "filter": {"op-type":"capture", "op-value":"time-based", "op-extra":{"period":"3"}}
                "requestId": "6578"
              }

[tguild]

I believe if you subscribe without any filters you get back all the signals your client application has access control to at whatever frequency the implementation provides, possibly excessive but expect rate limiting may be used to avoid overloading.

Thanks @erikbosch I agree besides what we require, there will be variation in implementations. Requesting a signal at a certain interval may not be met as you note and you will be provided it based on how often the implementation can produce or OEM policy permits perhaps even per signal. We know OEM vary widely in how often they broadcast on CAN.Too much variation dilutes the usefulness of standards but is the state of the industry.

good question on interval, assume or even recall ms, @UlfBj ?

[UlfBj]

Thanks @erikbosch for pointing out these things, that need more consideration.

The filter parameter is currently in the spec marked as Optional (ch. 5.2.2.3 in the Transport doc), but when thinking about it I tend to lean towards that it should be changed to Required. If not, then we need to define the server behaviour for when it is missing, to leave it to the implementation I think would lead to interoperability problems. The filtering should provide support for the different subscription behaviours that is of interest to clients, if the current filtering options do not cover a sufficient scope I think we should add what is missing. I do not support the idea that different server implementations have different behaviours, that would as I see it lead to interoperability issues.

Regarding the unit for the period in time based subscriptions it seems it is currently undefined. I believe it should be milliseconds. In the reference implementation I believe it is seconds at the moment, however. But that is easily changed.

[erikbosch]

If it is optional for VISS implementation to support filters, then of course it must be optional subscription request. But if filter support is mandatory, then it could theoretically be mandatory also in subscription requests.

If it is optional to specify filter in subscription requests, then I think that we at least should specify the default behavior if no filter is given. A possible solution could be to state that if no filter is given, then that corresponds to a time-based filter with period X, where X is implementation dependant, possibly configurable on server-level. (I.e. the requester should get the signal roughly every X millisecond, regardless of whether the value has changed or not)

(There will anyway be implementation limitations on what period you can request in a filter. )

[UlfBj]

If filtering is left as optional, then I think the proposal by @erikbosch is acceptable, with the addition that it shall be possible for the client to obtain the value X from the server. A possible solution to that could be that the client can retrieve X via a request for dynamic metadata, see spec for this feature. That would then require mandatory support for the latter.

[UlfBj]

I like the idea proposed by @gunnarx that all is mandatory, as we then do not have to add a solution for how to inform a client what is implemented. However, access control has the same "problem" with optional functionality, which then has to be solved without requiring the same solution as for optional filtering. Moving the responsibility for informing a client to the AGTS could possibly be seen as one. For the case of having optional parts, I then think that at least timebased filtering shall be mandatory, and that subscribe requests must include a filter expression.

[UlfBj]

Fixed in PR#437