Support OBD2 DBC file decoding and graphical plots

[MatinF]

I'm trying to use SavvyCAN to analyze a log file that contains regular OBD2 data frames (CAN ID 0x7e8).

Data and DBC I'm using below log file (converted to ASC) and OBD2 DBC file. The DBC file uses extended multiplexing. obd2-data-dbc.zip

Expected behavior I would expect that loading the OBD2 DBC file in SavvyCAN along with the OBD2 raw data would allow for interpreting the signals correctly, both in the trace view and when creating plots.

Trace view: Observed behavior The DBC appears valid when opened via CANDB++, Kvaser Database Editor etc. It's also working as intended in other tools such as asammdf. However, when I load the DBC file in SavvYCAN, I get an error warning regarding 2 signals that are invalid (though it is not clear from the error which ones).

Further, the trace view does show the full list of signals from the OBD2 DBC - though the expected behavior would be that only the relevant signal would show based on the multiplexing. For example, if a frame has a payload of 04 41 0C 16 6C FF FF FF then this should be interpreted as Engine Speed and the 4th and 5th data bytes should be used as part of the decoding.

Graph data: Observed behavior In the graph data view it is possible to create a signal for e.g. Engine Speed - but currently the graph view does not seem to support multiplexed signals. As a result, the plot is not usable for OBD2 signals - see e.g. below example of what it looks like. The result happens because all OBD2 response frames are getting decoded using the Engine Speed signal rules - yet only those frames for which the 3rd byte is 0C should be decoded as such (as per the multiplexing logic).

I believe resolving this would be useful for a large number of use cases!

[collin80]

You're right, the graphing system never got updated to properly support multiplexed signals so they show up with a lot of improper values from the other multiplexed messages. It is something I should correct. I'll leave this open until it is done.

[collin80]

I finally got a chance to look at this. The biggest issue is that I had no idea that the multiple layers of multiplexing could exist. When you try to load that DBC file it errors out on two lines.

SG_ ParameterID_Service01 m1M : 23|8@0+ (1,0) [0|255] "" VectorXXX SG_ ParameterID_Service02 m2M : 23|8@0+ (1,0) [0|255] "" VectorXXX

It appears it's legal to have:

SG_ service M : 11|4@0+ (1,0) [0|15] "" Vector__XXX

Be the top level multiplexor and then have those two as sub layers where they key on the value of the top level multiplexor then allow layers underneath to multiplex on them.

So, the issue was really my ignorance of the DBC file format. I do have an older copy of the DBC standard (or did) and I looked at other people's DBC loaders but I never saw anything that would suggest that multi-layer multiplexing was something you could do. I've never seen that in a DBC file before. Given the syntax it would appear that only two layers are feasible but maybe not? Can you just keep adding more and more layers by doing m1m2M or something like that? How would it know which layer is which then? Well, it appears I'm going to have to go have a long look at kdschlosser's code to figure this out. But, at least I know where it is going wrong.

[collin80]

Also, for reference:

https://assets.vector.com/cms/content/know-how/_application-notes/AN-ION-1-0521_Extended_Signal_Multiplexing.pdf

It seems Vector was nice enough to write up a description of how this all works. Thanks Vector!

[MatinF]

Hi Collin, we also provide a specific guide on the extended multiplexing with outset in the OBD DBC file:

https://www.csselectronics.com/screen/page/can-dbc-file-database-intro/language/en#dbc-advanced-topics

[collin80]

I've been working on supporting extended multiplexing. I didn't want to do it half way so it's taking quite a bit of time to pull off. At this point I can load and interpret the extended multiplexing and the editor interface shows it too. The in-memory representation is probably sufficient at this point to be able to use it for decoding but right now decoding is broken as I work on things. Having thought about it and looking at the examples I don't see any reason why extended multiplexing should be constrained to only two levels. So, my approach is simply hierarchical. You can nest as deeply as you want to (though I can't see more than 2-3 levels making any sense). I'm attaching a view from the main DBC screen below. You can see how everything is placed in the tree to show dependencies. "service" is the top level multiplexor, "ParameterID_Service01" is both multiplexed from "service" and a multiplexor for dependent signals. Then below it are all the signals that are multiplexed by ParameterID_Service01. It's possible to edit all of this in the program too. It doesn't save yet. It doesn't decode yet. But, loading and editing were the first order of business so I could load and visually confirm that things are being interpreted properly. It seems like extended multiplexing could be used to even have what would be "simple" multiplexed signals but instead give them a lower and upper bound value (simple multiplexing would ordinarily allow only a single value passed after m). So, I'll be supporting that too. If Vector doesn't actually allow that then... well, they don't. But, I will because it's pretty much required for extended multiplexing anyway. I suppose I should be careful not to allow DBC files to get saved with invalid configurations.

I don't know what Vector tools would think of it if I export a DBC where it uses simple multiplexing up until it has a SG_MULVAL that suddenly causes an otherwise simply multiplexed signal to have a range of values. It seems like it should be OK but perhaps I'll have to boot up Windows and go get a demo of Canalyzer so I can use CANDB++ to check things out. It's a shame that Vector doesn't have linux tools.

[MatinF]

Thanks Collin, appreciate your efforts on this. Happy to test out a DBC output from this via my CANDB++ installation if needed.

[collin80]

I just pushed my latest changes. Now it should more or less support extended multiplexing for loading, editing, saving, and for signal interpretation. Your DBC file and log file can now be properly interpreted. The code is still a bit ugly, there are likely many areas that are now a bit weird and need to be smoothed out. These changes make the DBC code essentially just an alpha or really early beta version. So, the continuous build should work but there are likely to be problems.

[MatinF]

Hi Collin, this is great, seems to work well on my end. As for the Graph Plots, is there a way to enable the multiplexing there or is that a separate effort to introduce?

Again, thanks for doing all this work to enhance the tool, it's really great! :-)

[collin80]

I will be fixing the code to support mulitplexed signals in the graphing window too. I hope to get to that some time next week.

[MatinF]

Awesome Collin, will make sure to test it once it's available

[MatinF]

Hi Collin, just wanted to check in on this to hear if it has been added in one the recent releases? happy to test it in that case

[mitchdetailed]

based on other comments, it might be now.

On Sun, Apr 4, 2021, 1:03 PM Martin Falch @.***> wrote:

Hi Collin, just wanted to check in on this to hear if it has been added in one the recent releases? happy to test it in that case

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/collin80/SavvyCAN/issues/271#issuecomment-813074380, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI62NLUDCB3A3UKTVJ452GLTHCSXTANCNFSM4WYWRETA .

[collin80]

I never did get it added to the graphing window. Multiplexed signals work pretty well in the main window now, even with extended multiplexing. But, this has not transitioned over to the graphing window yet. I didn't get time to implement it like I had hoped. Now I'm about a month behind where I thought I'd be. But, still, hopefully soon.

[MatinF]

Hi Collin, fully understand - really appreciate your great work on this!

[collin80]

And, soon finally happened. The current continuous build should be able to graph multiplexed signals properly.

[mitchdetailed]

extended multiplex dbc files?

On Wed, May 5, 2021, 6:23 PM Collin Kidder @.***> wrote:

And, soon finally happened. The current continuous build should be able to graph multiplexed signals properly.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/collin80/SavvyCAN/issues/271#issuecomment-833111071, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI62NLV4JDBGTH3GNI2Q34LTMHHPTANCNFSM4WYWRETA .

[collin80]

Well, I didn't test that. I don't have a whole lot of good test data to try out extended multiplexing. And, well, I forgot to try. I did test out a lot of simply multiplexed signals and it worked there. There is extended multiplexing support in code so it ought to work.

[MatinF]

Hi Collin, thanks for pushing this still!

I tested the new OBD2 plotting and it looks to work exactly as intended:

obd2-data.zip

A few minor comments/suggestions:

1) I love the new 'Scale to Data' feature that can be used in the plot window. Only change I'd recommend would be that it's "executed" by default when a new signal is added to the plot window to minimize #clicks for users

2) In the trace view, using the Interpret feature is a bit funky when using the OBD2 DBC case. I'm not sure if it can be changed, but as evident, it's a bit hard to identify the relevant piece of information (typically the signal value, e.g. 855 rpm, amongst the extensive text. It would be nice if the line with the signal value could be placed just below the data bytes so that expanding the row is not necessary to see this. It may be that I underestimate the complexity in determining which of the informations are relevant in this case, but since it seems like you're able to determine it in an intuitive way in the plot view, I figure the same logic could apply to determine the line to put at the top (below the data bytes) in the trace view - in the above screenshot it would be S1_PID_0C_EngineRPM: 855rpm etc

3) While not directly related to OBD2, the signal view could be further simplified for the case of J1939 DBC file interpretations. Specifically, the mask used is currently triggering based on the full 29-bit CAN ID when loading a J1939 signal - while in the trace view, it is based on the 18 bit PGN (as intended, and as specifiable when loading a J1939 DBC file in the DBC loader). It would be ideal that if a DBC signal is loaded from a J1939 DBC, the mask would not be 1FFFFFFF, but instead 3FFFF00. The result would be that the plot window would correctly display the J1939 signals without forcing the user to modify the mask or modifying the CAN ID. Hope this one makes sense.

Again, really appreciate your great work on this!