Use VSS as mapping format for dbcfeeder

[erikbosch]

This is an implementation of #41

Apart from changing the config there are also some general refactoring:

• Most python files moved to a dbcfeederlib folder. This is to simplify inclusion in unit testing
• Added some unit tests, it is still quite basic but at least some to verify syntax parsing
• Added a "on_change" config.

Testing fully performed:

• Sending signals to KUKSA.val Databroker (native and as Docker container)
• Sending signals to KUKSA.val Server (native and as Docker container)

Testing partially performed:

• elmcan: Verified that we reach and call the createvcan.sh script
• j1939: Exception observed, see #56

[erikbosch]

PR updated. Based on our discussions on printouts I created a short status printout, also showing max queue size so far. this would give at least some indication if your feeder works as expected

2023-02-14 17:37:31,844 INFO dbcfeederlib.databroker: Vehicle.Powertrain.Transmission.IsParkLockEngaged was already registered with type BOOLEAN
2023-02-14 17:37:31,847 INFO dbcfeederlib.databroker: Vehicle.Trailer.IsConnected was already registered with type BOOLEAN
2023-02-14 17:37:31,945 INFO dbcfeeder: Starting to process CAN signals
2023-02-14 17:37:31,948 INFO dbcfeeder: Number of VSS messages sent so far: 1, queue max size: 10
2023-02-14 17:37:31,950 INFO dbcfeeder: Number of VSS messages sent so far: 2, queue max size: 10
2023-02-14 17:37:31,955 INFO dbcfeeder: Number of VSS messages sent so far: 4, queue max size: 10
2023-02-14 17:37:31,965 INFO dbcfeeder: Number of VSS messages sent so far: 8, queue max size: 10
2023-02-14 17:37:32,060 INFO dbcfeeder: Number of VSS messages sent so far: 16, queue max size: 10
2023-02-14 17:37:32,374 INFO dbcfeeder: Number of VSS messages sent so far: 32, queue max size: 10
2023-02-14 17:37:33,056 INFO dbcfeeder: Number of VSS messages sent so far: 64, queue max size: 10
2023-02-14 17:37:34,444 INFO dbcfeeder: Number of VSS messages sent so far: 128, queue max size: 10
2023-02-14 17:37:37,255 INFO dbcfeeder: Number of VSS messages sent so far: 256, queue max size: 10
2023-02-14 17:37:42,844 INFO dbcfeeder: Number of VSS messages sent so far: 512, queue max size: 10
2023-02-14 17:37:53,989 INFO dbcfeeder: Number of VSS messages sent so far: 1024, queue max size: 10
^C2023-02-14 17:38:07,494 INFO dbcfeeder: Received signal 2, stopping...
2023-02-14 17:38:07,494 INFO dbcfeeder: Shutting down...

[SebastianSchildt]

Is this expected:

(dbc2val) scs2rng@RNG-C-001JT dbc2val % python --version
Python 3.8.13
(dbc2val) scs2rng@RNG-C-001JT dbc2val % python dbcfeeder.py
Traceback (most recent call last):
  File "dbcfeeder.py", line 45, in <module>
    from dbcfeederlib import dbc2vssmapper
  File "/Users/scs2rng/Documents/Dev/kuksa.val.feeders/dbc2val/dbcfeederlib/dbc2vssmapper.py", line 158, in <module>
    class Mapper:
  File "/Users/scs2rng/Documents/Dev/kuksa.val.feeders/dbc2val/dbcfeederlib/dbc2vssmapper.py", line 165, in Mapper
    mapping : dict[str, list[VSSMapping]] = {}
TypeError: 'type' object is not subscriptable

[SebastianSchildt]

Does not work on Python 3.8, needs 3.10 (did not test 3.9). Is this documented?

[SebastianSchildt]

Default loglevel seems to be Info when run locally but warning in container. Should probably be the same? Info for both, now that info is less noisy?

[erikbosch]

Our only statement on Python version is "Check that at least python version 3 is installed". In CI we use whatever Python-version is default on ubuntu-latest. Docker file use Python 3.9 so that 3.9 has been tested. I assume we likely can get it to run on Python 3.8, but we should better document what versions we intend to support. Python 3.11 is out - if we are to test all it will be a bit cumbersome to test all Python versions.

[SebastianSchildt]

I think for this specifically it is fine if we state 3.9 as minimum (I am not sure we would loose some typing features going back?).

I think with the current state of feeders, what we should test is, what we also put in the container. Just document, so people know what to expect

Was just surprised, as for the viss-client we made the choice 3.8 should be enough (there I think it makes sense)

[erikbosch]

There are some minor changes between Python 3.8 and 3.10, I sometimes experience them for vss-tools, where CI use Python 3.8 (and I use 3.10 when running locally). This stackoverflow describe the root cause also this time, it was easy to adapt to 3.8, but I will add something on supported versions.