unable to read mf4 with variable length of channels

[gdu2003]

Pyhton version

Python 3.5.3

Platform information

Windows 7

Numpy version

numpy = 1.14.0

mdfreader version

mdfdata.MDFVersionNumber : 400

Description

I have mf4 mdf file. It have more than 3000 signals or channels whatever it is called. All channels / signals are recorded at different frequencies as they are from CAN BUS recording of a car using ETAS INCA data logger. When i am trying to load it , it gives error saying structure must have same size. Looks like it is looking for same length of all signals. Where as in MDFREADER 0.2.7 it was all working fine. If i install MDFREADER 0.2.3 it works well. There seems to be some problem in 2.7.4

MDF Version of file is MF4.

mdf_file = filedialog.askopenfilename() mdfdata = mdfreader.mdf(fileName=mdf_file) Traceback (most recent call last):

File "", line 11, in mdfdata = mdfreader.mdf(fileName=mdf_file)

File "c:\python35\lib\site-packages\mdfreader\mdf.py", line 153, in init compression=compression)

File "c:\python35\lib\site-packages\mdfreader\mdfreader.py", line 399, in read convertAfterRead, filterChannelNames, compression)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 1248, in read4 buf.read(channelSet, info, self.fileName)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 340, in read self[recordID]['data'] = self.load(record, info, nameList=channelSet, sortedFlag=True)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 453, in load sortedFlag=sortedFlag, vlsd=vlsd)

ValueError: structures must have the same size

[ratal]

Hi There might be some regression between versions. The end of the error stack is missing maybe ? Could you inform what is line 453 to be sure ? If it is temps.update(temp), coudl reproduce this error and introduce a print of both temp and temps just before to better undersand the error ?

[gdu2003]

Following is the If block which has line 453 in mdfreader.py

401 temps = defaultdict() 402 # block header 403 temps.update(_loadHeader(self.fid, self.pointerTodata)) 404 if temps['id'] in ('##DL', b'##DL'): # data list block 405 temps.update(DLBlock(self.fid, temps['link_count'])) 406 if temps['dl_dl_next']: 407 index = 1 408 while temps['dl_dl_next']: # reads pointers to all data blocks (DT, RD, SD, DZ) 409 temp = defaultdict() 410 temp.update(_loadHeader(self.fid, temps['dl_dl_next'])) 411 temps['dl_dl_next'] = structunpack('<Q', self.fid.read(8))[0] 412 temps['dl_data'][index] = \ 413 structunpack('<{}Q'. 414 format(temp['link_count'] - 1), 415 self.fid.read(8 * (temp['link_count'] - 1))) 416 index += 1 417 if temps['dl_count']: 418 # read and concatenate raw blocks 419 buf = bytearray() 420 for DL in temps['dl_data']: 421 for pointer in temps['dl_data'][DL]: 422 # read fist data blocks linked by DLBlock to identify data block type 423 data_block = defaultdict() 424 data_block.update(_loadHeader(self.fid, pointer)) 425 if data_block['id'] in ('##DT', '##RD', b'##DT', 426 b'##RD', '##SD', b'##SD'): 427 buf.extend(self.fid.read( 428 data_block['length'] - 24)) 429 elif data_block['id'] in ('##DZ', b'##DZ'): 430 data_block.update(DZBlock(self.fid)) 431 data_block['data'] = \ 432 decompress_datablock( 433 self.fid.read(data_block['dz_data_length']), 434 data_block['dz_zip_type'], 435 data_block['dz_zip_parameter'], 436 data_block['dz_org_data_length']) 437 buf.extend(data_block['data']) 438 data_block['id'] = '##DT' # do not uncompress in DATABlock function 439 data_block['data'] = buf 440 temps['data'] = DATABlock(record, info, parent_block=data_block, channelSet=nameList, sortedFlag=sortedFlag) 441 else: # empty datalist 442 temps['data'] = None 443 elif temps['id'] in ('##HL', b'##HL'): # header list block for DZBlock 444 # link section 445 temps.update(HLBlock(self.fid)) 446 self.pointerTodata = temps['hl_dl_first'] 447 temps['data'] = self.load(record, info, zip=temps['hl_zip_type'], nameList=nameList, sortedFlag=sortedFlag) 448 elif temps['id'] in ('##DT', '##RD', b'##DT', b'##RD'): # normal sorted data block, direct read 449 temps['data'] = record.readSortedRecord(self.fid, self.pointerTodata, info, channelSet=nameList) 450 elif temps['id'] in ('##SD', b'##SD'): # VLSD 451 temps['data'] = self.fid.read(temps['length'] - 24) 452 temps['data'] = DATABlock(record, info, parent_block=temps, channelSet=nameList, sortedFlag=sortedFlag) 453 elif temps['id'] in ('##DZ', b'##DZ'): # zipped data block 454 temps.update(DZBlock(self.fid)) 455 temps['data'] = self.fid.read(temps['dz_data_length']) 456 temps['data'] = DATABlock(record, info, parent_block=temps, channelSet=nameList, sortedFlag=sortedFlag)

[ratal]

This is weird because temps['id'] in ('##DZ', b'##DZ') should not raise this kind of error, especially that is not so different from similar previous condition tests Are you sure you execute the same code ?

[ratal]

Long time no feedback, closing