MDF/MF4 merge/concatenate files of a series of measurement

[nexulm]

Python version

'python=3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]' 'os=Windows-10-10.0.18362-SP0' 'numpy=1.19.2' 'asammdf=6.2.0'

Code

MDF version

4.10

Code snippet

from asammdf import MDF

files = ['SVD_20210204_132557_00.mf4', 'SVD_20210204_135559_01.mf4'] mdf = MDF.concatenate(files,sync=False,direct_timestamp_continuation=True) mdf.save('concatenated.mf4', overwrite=True)

Traceback

Traceback (most recent call last): File ".\mdfconcate.py", line 5, in mdf = MDF.concatenate(files,sync=False,direct_timestamp_continuation=True) File "c:\Anaconda3\lib\site-packages\asammdf\mdf.py", line 1943, in concatenate cg_nr = merged.append( File "c:\Anaconda3\lib\site-packages\asammdf\blocks\mdf_v4.py", line 2777, in append ch = Channel(**kwargs) File "c:\Anaconda3\lib\site-packages\asammdf\blocks\v4_blocks.py", line 931, in init self.attachments = 1 AttributeError: 'Channel' object has no attribute 'attachments'

Description

I'm trying to merge/concatenate two or more mf4 files including references to video (.avi) files (Channel group 0-7). If I delete/filter Channel Group 0-8 out of the original files I'm able to concatenate these files but I like to concatenate the original files w/o the need of modifying the base files.

[danielhrisca]

hello, I believe this is fixed in the development branch

[nexulm]

I've used this command yesterday... pip install -I --no-deps https://github.com/danielhrisca/asammdf/archive/development.zip ...in Anaconda environment. But I'm not sure how to fix this error popping up regarding wheels dependencies easily: running build_ext building 'asammdf.blocks.cutils' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ ERROR: Failed building wheel for asammdf Failed to build asammdf ERROR: Could not build wheels for asammdf which use PEP 517 and cannot be installed directly

[danielhrisca]

you need to install the microsfot c++ build tools to compile the C extension

or

manually overwrite the asammdf code and delete the asammdf/blocks/cutils.pyd (this will fallback to the pure python code)

[nexulm]

Initial issue of this post is solved with the Dev branch... ...but another one has popped up now: CAN raw data in some of the Channel groups are not readable anymore.

[nexulm]

What's the difference between... from asammdf import MDF files = ['SVD_20210204_132557_00.mf4', 'SVD_20210204_135559_01.mf4'] mdf_conc = MDF(version='4.10') mdf_conc = MDF.concatenate(files) mdf_conc.save('concatenated_all6.mf4')

...and... from asammdf import MDF files = [MDF('SVD_20210204_132557_00.mf4', version='4.10'), MDF('SVD_20210204_135559_01.mf4', version="4.10")] mdf_conc = MDF(version='4.10') mdf_conc = MDF.concatenate(files) mdf_conc.save('concatenated_all6.mf4')

? The first code generate a concatenated file in MDF v4.00 which is not working for CANape as no buslogging is detected. The second one generates a MDF v4.10 concatenated file which is working as expected (with CAN logs). I'm using always the same two MF4 input files created in MDF v4.10 format.

[danielhrisca]

@nexulm please try 6.3.0.dev3

[nexulm]

Thanks, great! Now I can use the line... files = ['SVD_20210204_132557_00.mf4', 'SVD_20210204_135559_01.mf4'] ...again to get a concatenated MDF v4.10 file w/o additional options. :-) I've seen from 6.3.0.dev2 and 6.3.0.dev3 that always one empty (=0 byte) xxx.avi file is created and left on the filesystem after mdf.concatenate. Seems to be the .avi file which is referenced in the first Channel group but isn’t available in the directory while concatenating the mdf files. Any reason for that? Will the referenced .avi files concatenate as well if they are located on the same directory?

[danielhrisca]

Will the referenced .avi files concatenate as well if they are located on the same directory?

This kind of external attachment is not well supported if you modify the output file (cut, resample, concatenate)

[nexulm]

OK, nevertheless I'm currently confused by the size of the concatenated file, i.e.: Input: 6x xxx.mf4 file with 1,90 GB (2.049.966.080 Bytes) Output; 1x concatenate.mf4 with 2,78 GB (2.995.257.344 Bytes)

mdf_conc = MDF.concatenate(fmdflist) mdf_conc.save(outfile)

Any reason for the hugh amount of additional data? How can I avoid it?

[danielhrisca]

The external attachment is now transfered to an embedded attachment, hence the file size increase

[Hamzapyt]

Hello,

I have found that the concatenation has a limitation, How to drop channels if channel missing on the other files during the concatenation? the file refrence is files[0].

thank you,