ihedvall / mdflib

Implementation of the ASAM MDF data file.
https://ihedvall.github.io/mdflib/
MIT License
68 stars 29 forks source link

Problems converting to CSV using asammdf python lib #85

Closed okbuddy546 closed 1 month ago

okbuddy546 commented 4 months ago

Hi,

First of all, thank you for developing this project. I am currently using the can bus configuration to store can frames in mf4. I am encountering an issue where the generated mf4 doesn't seem to convert to csv when using assamdf's file conversion util in the python library. The generated mf4 also doesn't work with the CANedge mf4 converters. Just wondering if you have encountered such issues with the library in regards to csv conversion or if it's just an issue on my part 😓.

ihedvall commented 4 months ago

Well sort of. I previously fixed some missing flags so I suspect that there are some more issue. My implementation also have more "properties" than used by other bus loggers. I can check and try. The bus logging implementation is fairly new, so it may some bugs.

ihedvall commented 4 months ago

I have tested the CSS Electronics mdf2csv.exe (version 2.2.2) and it doesn't convert any MDF file created with MdfLib. It doesn't convert any of the standard MDF 4.1 and 4.2 files either but it do convert CanEdge log files.

I suspect that the conversion tool is somewhat sensitive towards CanEdge logging format. To solve the problem we might have to modify the CreateCanConfig function with an extra input that defines "CanEdge format", "Vector format".

Alternative, I can make a mdf2csv command line application or add this functionality to the MdfViewer.

I have tested the AsamMdf GUI CSV conversion and it seems OK but I assume there is some other command line call that you use. Can you give me a hint about the procedure.

okbuddy546 commented 4 months ago

Ah slight correction, the generated mf4 files do seem to generate correct csv files with both the AsamMdf Gui and the AsamMdf API. I had put the wrong dbc config in when testing with the API resulting in no csv being generated.

I think your right in that the conversion tool is specific to CANedge format which is why the converstion tools don't work. Having an option to choose between the current can config with the additional properties and other formats such as "CANedge format" sounds like a good idea in the case that users are specifically using CANedge devices and services.

ihedvall commented 4 months ago

Well, life is not easy. I tried some modification on the MdfWriter CAN configuration but no luck. There is actual very little changes so I hoped that the converter at least gave some hint about the problem. Finding a working MDF file format that works might require some major test and try work. It would be nice if it was possible to debug the mdf2csv software to check what it fails on.

If you want to parse out DBC signal values from an MDF file, you can use the DbcLib + MdfLib. In The DbcViewer application you can open an DBC file and load in a MDF bus log file. The AsamMdf do have the same function though.

Please let me know if you have any idea of converter needs. There is an upcoming ReportExplorer application that target these type of problems.

Best Regards Ingemar Hedvall

ihedvall commented 3 months ago

I think the original AsamMdf problem is fixed but the CSS Electronics converter (mdf2csv.exe) problem is not an easy fix. The simplest solution is to make a mdf2csv.exe that works with all bus loggers. Below are the requirements.

  1. Convert functionality according to the existing functionality i.e. it should handle CSS log files.
  2. Add DBC file handling so the end user can extract DBC signal values from the CAN frame data.
  3. Add functionality to handle non bus logger files and multiple CG blocks. Note that there exist 2 different way to handle this. Multiple CSV files or a Single (time merged) file.
ihedvall commented 2 months ago

@okbuddy546 I have added a preliminary mdf2csv application according to the above. It's not 100% compatible with the CSS converter but give it a try if you have the time.

ihedvall commented 1 month ago

I close this issue as the DBC Viewer has similar functionality as the ASAM MDF GUI regarding converting MDF log files to DBC signal values. The new mdf2csv application can replace the CSS version.