MDF4 signalDataType 13

[kkreplin]

When I read in my mf4 data I get the following error:

lib/python2.7/site-packages/mdfreader/mdf4reader.py in read4(self, fileName, info, multiProc, channelList, convertAfterRead, filterChannelNames)
   1303                                             buf[recordID]['data'] is not None: # no data in channel group
   1304                                         if isinstance(buf[recordID]['data'],recarray):
-> 1305                                             temp = buf[recordID]['data'].__getattribute__(convertName(recordName))  # extract channel vector
   1306                                         else:
   1307                                             temp = buf[recordID]['data'][convertName(recordName)]
lib/python2.7/site-packages/numpy/core/records.pyc in __getattribute__(self, attr)
    444             res = fielddict[attr][:2]
    445         except (TypeError, KeyError):
--> 446             raise AttributeError("recarray has no attribute %s" % attr)
    447         obj = self.getfield(*res)
    448 

AttributeError: recarray has no attribute Analog: Time_abs_title

It is caused by having besides a normal entry Time with signalDataType 0 an additional entry Time_abs with signalDataType 13. The class recordChannel in mdf4reader.py however doesn't handle this signalDataType. When I manually set the signalDataType of Time_abs to 0 (like Time), the data can be read in, however, I get the following error:

lib/python2.7/site-packages/mdfreader/mdf4reader.py in load(self, record, zip, nameList, sortedFlag)
    465             temps['data'] = self.load(record, zip=temps['hl_zip_type'], nameList=nameList, sortedFlag=sortedFlag)
    466         elif temps['id'] in ('##DT', '##RD', b'##DT', b'##RD'):  # normal sorted data block, direct read
--> 467             temps['data'] = record.readSortedRecord(self.fid, self.pointerTodata, channelList=nameList)
    468         elif temps['id'] in ('##SD', b'##SD'):  # VLSD
    469             temps['data'] = self.fid.read(temps['length'] - 24)

lib/python2.7/site-packages/numpy/core/records.pyc in fromfile(fd, dtype, shape, offset, formats, names, titles, aligned, byteorder)
    781     nbytesread = fd.readinto(_array.data)
    782     if nbytesread != nbytes:
--> 783         raise IOError("Didn't read as many bytes as expected")
    784     if name:
    785         fd.close()

IOError: Didn't read as many bytes as expected

Do you have any suggestion on how to handle the Time_abs entry with signalDataType 13 properly?

Thank you.

[ratal]

Thanks for already digging this concern. signalDataType 13 is canopen date. record channel class do not handle it directly. mdf4reader is actually creating channels instead for ms, min, hour, day, month, year in loadinfo() method from record class Using the reference files I have using signalDataType 13, there is no concern. Your file might have a combination of options or variants that I did not experience or think of. Just with this post, I unfortunately do not have enough information to support you more. Can you dig further from what I explained or send me a sample of your file ?

[ratal]

I was digging a bit more on your concern. I do not understand why you have an error at line with getattribute looking for channel Time_abs. This channel should not exist but this line of code is within a "if" condition excluding looking for this channel of signalDataType 13 or 14 (CANopen), you can read this a few lines upper. Could you do a small test by printing chan.signalDataType and chan.name at the error line ? By the way, line number does not correspond with the master I have -> which version are you using ?

[kkreplin]

Hi,

thanks for your quick answer. I'm also working with the master, but had already some prints added to the mdf4reader.py so I got different line numbers. Moving back to the master and just adding the print 'signalDataType:', chan.signalDataType, ', chan.name:', chan.name, ', recordName:', recordName at the error line I get:

lib/python2.7/site-packages/mdfreader/mdf4reader.py in read4(self, fileName, info, multiProc, channelList, convertAfterRead, filterChannelNames)
   1298                                         print 'signalDataType:', chan.signalDataType, ', chan.name:', chan.name, ', recordName:', recordName
   1299                                         if isinstance(buf[recordID]['data'],recarray):
-> 1300                                             temp = buf[recordID]['data'].__getattribute__(convertName(recordName))  # extract channel vector
   1301                                         else:
   1302                                             temp = buf[recordID]['data'][convertName(recordName)]

lib/python2.7/site-packages/numpy/core/records.pyc in __getattribute__(self, attr)
    444             res = fielddict[attr][:2]
    445         except (TypeError, KeyError):
--> 446             raise AttributeError("recarray has no attribute %s" % attr)
    447         obj = self.getfield(*res)
    448 

AttributeError: recarray has no attribute Analog: Time_abs_title

the same error. The print output is:

signalDataType: 0 , chan.name: Analog: Time0 , recordName: Analog: Time0
signalDataType: 0 , chan.name: CPU-Auslastung , recordName: Analog: Time_abs

So, there is a mismatch between the chan.name and the recordName here. Thanks for any further help!

[ratal]

Thanks, progressing. In order to better understand what is going on, could you show the problematic file with mdfvalidator from vector, especially in the datagroup and channelgroup containing theses Time0, CPU-Auslastung and Time_abs ? mdfvalidator Unfortunately, I realized the reference file I use to test mdfreader against CanOpen Date or Time does not contains any other channels than this kind of specific channel. I might have missed somewhere interaction of CanOpen channel with other channels in same data group --> A not sensitive file example would help

[kkreplin]

Dear Aymeric,

thank you for your answer! Unfortunately, the file is sensitive. Do you have any experience in stripping/anonymisation of an mf4 file? I would really like to provide you with a working example file, which just contains the necessary information.

Using the mdfvalidator I found out, that the interplay of Time_abs and CPU-Auslastung is contained in Data Group 4. Maybe it is possible to just keep this Data Group 4 and discard the rest of the file?

If you think that's not possible, can I provide you with any other debugging information? Thank you very much for your help!

Katharina

[ratal]

I ended up adding a new class for CANOpen channels. Please pull latest version and try.

[kkreplin]

Dear Aymeric,

I tested your solution: it works! Thank you very much for the fix.

Katharina