ratal / mdfreader

Read Measurement Data Format (MDF) versions 3.x and 4.x file formats in python
Other
169 stars 73 forks source link

Int or float value instead of string in conversion? #176

Closed rolfub closed 4 years ago

rolfub commented 4 years ago

Screenshot

Hello Aymeric, I have a question. Is it possible to get a float or integer value instead of "Error_ORH" when using conversion? (see also Screenshot.png) Yours sincerely, Rolf

ratal commented 4 years ago

Hi, Your channel is unicode, not integer, even the u'3902.0'. I guess you could use convert_after_read=False to get raw data by looking at the 'data' key value and change the channel conversion to your needs.

rolfub commented 4 years ago

Hello Aymeric, thank you for your answer. Now I see that I haven't wrote it correctly. :-( I meant that "Error_ORH" is a problem for plotting the values. u"3902.0" for example is not that critical. You can convert it to a float value before plotting but that is not possible with "ERROR_ORH". So, is it possible to get a value instead of a name? Yours sincerely, Rolf

ratal commented 4 years ago

Hi, These unicode are most probably not recorded in the mdf but numbers instead that are later converted based on conversion channel type and content. If you read the file with convert_after_read=False argument, each channel's ['data'] will be kept not converted along with its conversion channel to convert the data 'on demand', basically if get_channel_data() is called. It is more memory efficient and conversion generally requires lot of computation. For instance, if you consider time, it is probably an unsigned integer 32 that is then converted by multiplying to a floating (sampling time), giving a float64, occupying twice memory. In your case, you could read the file without conversion, look at the conversion channel and replace the Error_ORH by what you want, like zero. If it is value to text conversion, it is most probably inside the cc_ref list. This might ask too much knowledge of mdf so an an alternative is to use MaskedArray by creating your own mask, then you should have no issue while plotting. It could be like this (not tested): data = data.view(MaskedArray) data = masked_equal(u'Error_ORH', data)

rolfub commented 4 years ago

Hello Aymeric, my colleagues means that I haven't explained it correctly to you, so he creates some screenshots to show his problem.

He not only wants to plot the values but to use the values for calculations. So he needs all values in integer or float format (-> screenshot):

grafik

To get this values he does some modifications to your code (-> screenshot).

grafik grafik

He wants me to ask you if there is a possibility to create a solution for him because he does not want to modificate your code.

Yours sincerely,

Rolf

ratal commented 4 years ago

Same as my previous comment, use convert_after_read parameter : yop=mdfreader.Mdf('yourfilename', convert_after_read=False) _value_to_text_conversion() will not be called and you will be able to access "raw" data with yop['channelName]['data'] (not .get_channel_data() ) To keep it as is, you could simply remove the conversion class below yop['channelName'].pop('conversion') to use .get_channel_data() later without conversion

danielhrisca commented 4 years ago

@ratal I think you mean _get_channeldata

ratal commented 4 years ago

thanks Daniel :)

rolfub commented 4 years ago

Hello Aymeric,

you are wright, I will get then the raw data without conversion. But my colleagues needs the conversion with the data from cc_ref to get the scaled values of bus signals.

Yours sincerely,

Rolf

ratal commented 4 years ago

Hi, You can modify the CCBlock conversion block class contained in yop['channelName']['conversion'] the way you want. It contains the cc_ref, you can modify it. You could even change type of conversion if you wish.

ratal commented 4 years ago

I guess you could succeed, closing