Question: How to convert arbitrary CAN bus logging formats to decoded MDF?

[driftregion]

Python version

Please run the following snippet and write the output here

'python=3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]'
'os=Linux-6.8.0-40-generic-x86_64-with-glibc2.35'
'numpy=1.24.1'
ldf is not supported
xls is not supported
xlsx is not supp

Code

Code snippet

#!/usr/bin/env python3

import asammdf
import can
import cantools
import pandas as pd
import sys
import re
import tempfile
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("files", nargs="+", type=str)
parser.add_argument("dbc", type=str)
args = parser.parse_args()
db = cantools.db.load_file(args.dbc)
unknown_ids = []
for logfile in args.files:
    logs = {}
    with can.LogReader(logfile) as reader:
        for frame in reader:

            try:
                db_msg = db.get_message_by_frame_id(frame.arbitration_id)
                decoded = db_msg.decode(frame.data)
            except KeyError:
                if frame.arbitration_id not in unknown_ids:
                    print(f'Unknown CAN_ID: {hex(frame.arbitration_id)}', file=sys.stderr)
                    unknown_ids.append(frame.arbitration_id)
                continue
            except ValueError:
                print(f'CAN_ID decode error: {hex(frame.arbitration_id)}', file=sys.stderr)
                print(frame.data)
            decoded['time'] = frame.timestamp

            try:
                log = logs[frame.arbitration_id]
            except KeyError:
                log = []
                logs[frame.arbitration_id] = log

            log.append(decoded)

    with asammdf.MDF() as mdf:
        for can_id, log in logs.items():
            df = pd.DataFrame(log)
            df = df.set_index('time')
            db_msg = db.get_message_by_frame_id(can_id)
            mdf.append(df, comment=db_msg.name, units={sig.name: sig.unit for sig in db_msg.signals})

        basename = logfile.split('.')[0]
        fname = f'{basename}.mdf'
        print(mdf.save(fname))

Traceback

Traceback (most recent call last):
  File "/home/user/scripts/to_mdf.py", line 55, in <module>
    print(mdf.save(fname))
  File "/home/user/repos/asammdf/src/asammdf/blocks/mdf_v4.py", line 9551, in save
    address = channel.to_blocks(address, blocks, defined_texts, cc_map, si_map)
  File "/home/user/repos/asammdf/src/asammdf/blocks/v4_blocks.py", line 1056, in to_blocks
    text=text.encode("utf-8", "replace"),
AttributeError: 'NoneType' object has no attribute 'encode'

Description

Hello, thanks for asammdf. I am a GUI user, converting various CAN log formats (.asc, .trc, .log, .csv, ...) with DBC to MDF for viewing in asammdf. I used to use the above script, but it is broken in recent releases of asammdf. Is there an existing tool or technique for this workflow?

[danielhrisca]

make sure that sig.unit is not None

[driftregion]

make sure that sig.unit is not None

Fantastic idea!

            mdf.append(df, comment=db_msg.name, units={
                sig.name: sig.unit if sig.unit else "" for sig in db_msg.signals
            })

Lastly, I ran into this issue:

2024-11-18, 16:58:44
--------------------------------------------------------------------------------
<class 'TypeError'>  
'NoneType' object is not subscriptable
--------------------------------------------------------------------------------
  File "/home/user/.local/bin/asammdf", line 8, in <module>
    sys.exit(main())
  File "/home/user/repos/asammdf/src/asammdf/gui/asammdfgui.py", line 45, in main
    _main_window = MainWindow(measurements or args.measurements)
  File "/home/user/repos/asammdf/src/asammdf/gui/widgets/main.py", line 760, in __init__
    self._open_file(name)
  File "/home/user/repos/asammdf/src/asammdf/gui/widgets/main.py", line 1184, in _open_file
    widget = FileWidget(
  File "/home/user/repos/asammdf/src/asammdf/gui/widgets/file.py", line 538, in __init__
    buses = lin_databases[::2]

which I fixed with

rm ~/.config/py-asammdf/py-asammdf.conf

Thanks @danielhrisca