ratal / mdfreader

Read Measurement Data Format (MDF) versions 3.x and 4.x file formats in python
Other
169 stars 74 forks source link

error in save method when noDataLoad=True #96

Closed danielhrisca closed 6 years ago

danielhrisca commented 6 years ago

Environment

* 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]
* Windows-10-10.0.14393-SP0
* Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
* 16GB installed RAM
* mdfreader 0.2.6

Files used

I have changed the test files with the new ones found here:

https://github.com/danielhrisca/asammdf/blob/master/benchmarks/test%20files.7z

MDF version 3 error

Save file : mdfreader 0.2.6 noDataLoading mdfv3 --> Excption during run:
  File "E:\02__PythonWorkspace\asammdf\benchmarks\bench.py", line 437, in save_reader3_nodata
    x.write(r'x.mdf')
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfreader.py", line 422, in write
    self.write3(fileName=self.fileName)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf3reader.py", line 1403, in write3
    fid.write(fromarrays(dataList).tobytes(order='F'))
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\numpy\core\records.py", line 619, in fromarrays
    raise ValueError("array-shape mismatch in array %d" % k)

MDF version 4 error

Save file : mdfreader 0.2.6 noDataLoading mdfv4 --> Excption during run:
  File "E:\02__PythonWorkspace\asammdf\benchmarks\bench.py", line 319, in save_reader4_nodata
    x.write(r'x.mf4')
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfreader.py", line 420, in write
    self.convertAllChannel()
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdfreader.py", line 455, in convertAllChannel
    return self._convertAllChannel4()
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1643, in _convertAllChannel4
    self.read4(self.fileName, convertAfterRead=True)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1536, in read4
    self._convertAllChannel4()
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1646, in _convertAllChannel4
    [self._convertChannel4(channelName) for channelName in self]
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1646, in <listcomp>
    [self._convertChannel4(channelName) for channelName in self]
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1634, in _convertChannel4
    self.setChannelData(channelName, self._getChannelData4(channelName))
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1561, in _getChannelData4
    self.read4(fileName=None, info=self._info, channelList=[channelName], convertAfterRead=False)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 1449, in read4
    buf.read(channelSet)  # reads raw data from data block with DATA and DATABlock classes
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 364, in read
    self[recordID]['data'] = self.load(record, zip=None, nameList=channelSet, sortedFlag=True)
  File "e:\WinPython-32bit-3.6.1.0Qt5\python-3.6.1\lib\site-packages\mdfreader\mdf4reader.py", line 456, in load
    raise Exception('unknown data block')
ratal commented 6 years ago

I shoul dhave this in last commit.

danielhrisca commented 6 years ago

MDF 3

 Save file : mdfreader 0.2.7 noDataLoading mdfv3
  File "E:\02__PythonWorkspace\asammdf\benchmarks\bench.py", line 408, in save_reader3_nodata
    x.write(r'x.mdf')
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdfreader.py", line 431, in write
    self.write3(fileName=fileName)
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf3reader.py", line 1409, in write3
    fid.write(fromarrays(dataList).tobytes(order='F'))
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\numpy\core\records.py", line 619, in fromarrays
    raise ValueError("array-shape mismatch in array %d" % k)

MDF 4

Save file : mdfreader 0.2.7 noDataLoading mdfv4
  File "E:\02__PythonWorkspace\asammdf\benchmarks\bench.py", line 448, in save_reader4_nodata
    x.write(r'x.mf4')
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdfreader.py", line 429, in write
    self.convertAllChannel()
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdfreader.py", line 464, in convertAllChannel
    return self._convertAllChannel4()
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1309, in _convertAllChannel4
    self.read4(self.fileName, convertAfterRead=True)
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1202, in read4
    self._convertAllChannel4()
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1312, in _convertAllChannel4
    [self._convertChannel4(channelName) for channelName in self]
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1312, in <listcomp>
    [self._convertChannel4(channelName) for channelName in self]
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1300, in _convertChannel4
    self.setChannelData(channelName, self._getChannelData4(channelName))
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1227, in _getChannelData4
    self.read4(fileName=None, info=self.info, channelList=[channelName], convertAfterRead=False)
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 1107, in read4
    buf.read(channelSet, info)  # reads raw data from data block with DATA and DATABlock classes
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 371, in read
    self[recordID]['data'] = self.load(record, info, zip=None, nameList=channelSet, sortedFlag=True)
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 454, in load
    temps['data'] = record.readSortedRecord(self.fid, self.pointerTodata, info, channelSet=nameList)
  File "E:\WinPython-64bit-3.6.1.0Qt5\python-3.6.1.amd64\lib\site-packages\mdfreader\mdf4reader.py", line 805, in readSortedRecord
    rec = zeros((self.numberOfRecords, ), dtype=numpyDataRecordFormat)
ratal commented 6 years ago

Hi Daniel, Sorry, I missed your comment that you changed test files and could explain why I could not catch your exceptions. But unfortunately, you had to remove it from you repository. Can you make it available for me again please ? Thanks

danielhrisca commented 6 years ago

Hello Aymeric,

I will attach the new files in the morning.

danielhrisca commented 6 years ago

I've uploaded them to the issue in my repo

https://github.com/danielhrisca/asammdf/issues/14

ratal commented 6 years ago

It should be fixed with last commit