Various issues related to Bus Logging

[MatinF]

Hi Daniel

We are currently in the process of re-structuring our MDF file, and have come across a few issues with asammdf.

Sample data related to the below can be found here: sample-data.zip

We also have a version that resolves the issues here (though there may be better way to implement the resolutions): asammdf-7.1.1.post0.dev0-cp39-cp39-win_amd64.zip

Timestamps are not read using the description in the channel

Previously we encoded our timestamps as 8 byte floating point values. We have now changed this to an unsigned integer with scaling via a CC block, which allows us to reduce the precision/number of bits of information needed. If we reduce the timestamp channel from 8 to 6 bytes, asammdf still reads 8 bytes worth of data from the file, thus creating garbage timestamps, as it mixes other data into the timestamp data.

Missing coercion to integers

For CAN, we can guarantee that the value of the DataLength column always can be derived from the DLC column. This would allow us to drop one of the column, and use a conversion rule to map it as a virtual channel instead.

For the DataLength column, this is an issue, since the result of the conversion rule always is a float, and floats cannot be used to create slices/access array data by index.

It might make sense for BusLogging to always cast the values to (unsigned) integers. Whether a general rule can be made for MDF, based on either a lack of unit after the conversion or some XML metadata is up for debate.

Missing columns in trace view

Trace view should ideally show all populated/all possible BusLogging columns. Missing in CAN buslogging are:

• EDL
• BRS
• ESI
• (Perhaps more)

For LIN BusLogging, it might be beneficial to show the data for checksum error frames if present.

Records with the same time stamp but different channels in trace view

When using multiple channels to store records from different sources as one source per channel, the merged trace view is unable to show records with identical timestamps (As when they are received at the same time on multiple source interfaces). Looking at the data in each dedicated group works fine.

[danielhrisca]

Timestamps are not read using the description in the channel

Previously we encoded our timestamps as 8 byte floating point values. We have now changed this to an unsigned integer with scaling via a CC block, which allows us to reduce the precision/number of bits of information needed. If we reduce the timestamp channel from 8 to 6 bytes, asammdf still reads 8 bytes worth of data from the file, thus creating garbage timestamps, as it mixes other data into the timestamp data.

Please try the fix in the development branch

[MatinF]

Thanks Daniel, that issue is now resolved

[danielhrisca]

Records with the same time stamp but different channels in trace view

When using multiple channels to store records from different sources as one source per channel, the merged trace view is unable to show records with identical timestamps (As when they are received at the same time on multiple source interfaces). Looking at the data in each dedicated group works fine.

The whole code logic dealing with the trace is assuming no duplicated timestamps in the output view

[MatinF]

Hi Daniel, yes we realize that one is tricky - not sure if it can be resolved.

We of course ensure (as per the MDF standard) that we have no overlapping timestamps within a given Channel Group.

However, the MDF standard does not prohibit overlapping timestamps across Channel Groups. In practice, this may happen in devices like ours with 50 microsecond resolution. Below is an example of the implication:

Perhaps a solution would be to use the 'Index' column as the index in the trace views instead of the timestamps column?

We tested this as well in CANalyzer, which is able to display both messages despite the overlapping timestamps.

[danielhrisca]

The duplication solution was not so hard, but now there is a bigger chance of out of memory errors for the CAN and LIN traces. Please try it out

[MatinF]

Hi Daniel, thanks - it looks like it works as it should, great job!

[danielhrisca]

For LIN BusLogging, it might be beneficial to show the data for checksum error frames if present.

this is shown in the Details column

[MatinF]

Thanks Daniel, that works as intended as well. One observation is that the sample data does actually have the LIN databytes for that particular Checksum Error Frame, which would be nice to display in the Data Bytes column if possible. It displays correctly if we use the tabular display for the specific channel group related to the LIN Checksum Errors (CH64).

[danielhrisca]

Missing columns in trace view

Trace view should ideally show all populated/all possible BusLogging columns. Missing in CAN buslogging are:

• EDL
• BRS
• ESI
• (Perhaps more)

Please check this as well with the new development branch code

[danielhrisca]

Thanks Daniel, that works as intended as well. One observation is that the sample data does actually have the LIN databytes for that particular Checksum Error Frame, which would be nice to display in the Data Bytes column if possible. It displays correctly if we use the tabular display for the specific channel group related to the LIN Checksum Errors (CH64).

Done

[MatinF]

Great Daniel!

I think that solves most of it - I'll check tomorrow with the team on the "Missing coercion to integers".

One additional related issue we found during the tests:

In bus logging view, the conversion rules are not applied for the DataLength field. While this works for regular CAN frames for DLC 0 to 8, it poses an issue for CAN FD frames, where there is a non-linear relationship. If the DataLength is stored as a virtual channel, based on the DLC value, too few bytes are shown in CAN bus logging. E.g. for a DLC of 0xE or 14, the DataLength field should be 48, but since the raw value is used, only 14 of the 48 bytes are shown. See e.g. below:

[danielhrisca]

It is fixed in the development branch

[MatinF]

Great Daniel, awesome work as always. I believe this concludes all items.