search

ratal / mdfreader

Read Measurement Data Format (MDF) versions 3.x and 4.x file formats in python
Other
169 stars 74 forks source link

lazy loading for raw values #123

Closed ginseil closed 6 years ago

ginseil commented 6 years ago

Hi,

I have mdf files with a size in GB and need lazy access to raw values and the calculated ones.

I know I can access the calculated ones using the noDataLoading = True flag but I have to open the file with convertAfterRead=False to access the raw values. Since I want to process the mdf file channel by channel I'll end up reading the file once again each time I need access to a channel's the raw values...

Could you add the possibility to provide raw values on demand?

danielhrisca commented 6 years ago

Hello, why do you need the raw values?

ginseil commented 6 years ago

We export them next to the calculated ones.

ratal commented 6 years ago

Yes, possible. By when you need it ? (vacation period coming)

ginseil commented 6 years ago

Hmm, probably 15. january...

ratal commented 6 years ago

I must admit I never considered and tested combination of both argument noDataLoading and convertAfterRead, but I did a few tests and seems to be working properly, at least I can get back converted channels. I guess you want to extract both raw and converted data from getChannelData after reading with noDataLoading=True, so I will add it an argument specificly for returning only raw data.

ginseil commented 6 years ago

That would be awesome. Thanks!

ratal commented 6 years ago

I implemented the argumetn raw_data flag in getChannelData for last commit, you can give a try.

ginseil commented 6 years ago

I tried it and it works like a charm. Thanks!