error saving MDF version 4 test file

[danielhrisca]

using commit https://github.com/ratal/mdfreader/commit/c7d18601b491b22714ab9adc5603cdf24460da17

using bench script https://github.com/danielhrisca/asammdf/blob/development/benchmarks/bench.py

I get this error when saving mdf version 4 file:

Traceback (most recent call last):
  File "d:\WinPython-64bit-3.6.2.0Qt5b3\python-3.6.2rc1.amd64\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "d:\WinPython-64bit-3.6.2.0Qt5b3\python-3.6.2rc1.amd64\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\Work\asammdf\benchmarks\bench.py", line 158, in save_reader4
    x.write(r'x.mf4')
  File "d:\WinPython-64bit-3.6.2.0Qt5b3\python-3.6.2rc1.amd64\lib\site-packages\mdfreader\mdfreader.py", line 421, in write
    self.write4(fileName=self.fileName)
  File "d:\WinPython-64bit-3.6.2.0Qt5b3\python-3.6.2rc1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1649, in write4
    temp.write(fid)
  File "d:\WinPython-64bit-3.6.2.0Qt5b3\python-3.6.2rc1.amd64\lib\site-packages\mdfreader\mdfinfo4.py", line 307, in write
    fid.write(pack('<8s8s8s4sH30s2H', *head))
struct.error: argument for 's' must be a bytes object

[ratal]

Please check but it should be fixed with latest commit along with some small saving time improvements.

[danielhrisca]

Hi Aymeric,

I'm afraid there are still issues with writing mdf4 files:

Process Process-20:
Traceback (most recent call last):
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "E:\02__PythonWorkspace\asammdf\benchmarks\bench.py", line 156, in save_reader4
    x.write(r'x.mf4')
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfreader.py", line 423, in write
    self.write4(fileName=self.fileName)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1653, in write4
    pointers.update(temp.write(fid))
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfinfo4.py", line 345, in write
    currentPosition = _writeHeader(fid, '##HD', 104, 6)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfinfo4.py", line 208, in _writeHeader
    fid.write(HeaderStruct.pack(*head))
struct.error: argument for 's' must be a bytes object

[ratal]

Hi Daniel, Yes, happened in python3 not python2, thanks. Added small b'' Should also be ok for python 3

[danielhrisca]

Hello Aymeric,

the issue is fixed and mdfreader works faster when saving

Benchmark environment

• 3.6.2rc1 (heads/3.6:268e1fb, Jun 17 2017, 19:01:44) [MSC v.1900 64 bit (AMD64)]
• Windows-10-10.0.15063-SP0
• Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
• 16GB installed RAM

Notations used in the results

• nodata = MDF object created with load_measured_data=False (raw channel data not loaded into RAM)
• compression = MDF object created with compression=blosc
• compression bcolz 6 = MDF object created with compression=6
• noDataLoading = MDF object read with noDataLoading=True

Files used for benchmark: * 183 groups * 36424 channels

Open file	Time [ms]	RAM [MB]
mdfreader 0.2.6 mdfv3	3698	542
mdfreader 0.2.6 compression mdfv3	5041	262
mdfreader 0.2.6 noDataLoading mdfv3	1933	193
mdfreader 0.2.6 mdfv4	42596	1315
mdfreader 0.2.6 compression mdfv4	46789	1027
mdfreader 0.2.6 noDataLoading mdfv4	5001	948

Save file	Time [ms]	RAM [MB]
mdfreader 0.2.6 mdfv3	7925	541
mdfreader 0.2.6 mdfv4	4065	1315

Get all channels (36424 calls)	Time [ms]	RAM [MB]
mdfreader 0.2.6 mdfv3	90	541
mdfreader 0.2.6 mdfv4	97	1315