adamreeve / npTDMS

NumPy based Python module for reading TDMS files produced by LabView
http://nptdms.readthedocs.io
GNU Lesser General Public License v3.0
237 stars 88 forks source link

Unable to open file #9

Closed flatzo closed 11 years ago

flatzo commented 11 years ago

Loading a TDMS file generated with NI-DAQmx with the following

from nptdms import TdmsFile
tdms_file = TdmsFile("example.tdms")

Gives the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\nptdms\tdms.py", line 148, in __init__
    self._read_segments(tdms_file)
  File "C:\Python27\lib\site-packages\nptdms\tdms.py", line 160, in _read_segmen
ts
    previous_segment)
  File "C:\Python27\lib\site-packages\nptdms\tdms.py", line 346, in read_metadat
a
    segment_obj._read_metadata(f)
  File "C:\Python27\lib\site-packages\nptdms\tdms.py", line 655, in _read_metada
ta
    self.data_type.length * self.dimension)
TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

However, the file can be easily read using C and NI's DLL.

adamreeve commented 11 years ago

It looks like the file uses a data type that isn't supported yet, either extended floating point values or DAQmx raw data (I'm guessing the latter if it came from DAQmx). This should probably be caught earlier and give a more helpful error message.

It would help if you can upload an example TDMS file that doesn't work somewhere, then I can see what type of data it is and get an idea of how easy it would be to fix this. A description of the data you get using NI's DLL would be helpful too.

flatzo commented 11 years ago

It doesn't seem to be raw data. When importing it to excel, it shows as DT_DOUBLE so I read it as normal C double using the DLL.

I uploaded the TDMS file here. It is a simple 1 voltage channel dump.

adamreeve commented 11 years ago

Yeah your file uses DAQmx raw data, and it looks like this would be very tricky to support. The NI article on the TDMS file format structure at http://www.ni.com/white-paper/5696/en/ says:

This article does not describe how to decode DAQmx data. If you need to read a TDMS file with software that implements native support for TDMS (without using any components provided by National Instruments), you will not be able to interpret this data.

I couldn't find anything promising on the NI forums on how to read this data either, eg. http://forums.ni.com/t5/LabVIEW/TDMS-DAQmx-decode/td-p/2172986. From that thread:

All that being said, to answer your second question, the raw data in a DAQmx TDMS file simply consists of data in the same exact format that you get if you were to call DAQmx Read Raw. The output is a binary blob of interleaved channel data from the ADC without calibration or scaling. The process of deinterleaving that data and applying all of the necessary information with the exact numerical precision is fairly complex and error-prone if left to every user. In many ways, thi is why we created the built-in TDMS logging feature into DAQmx - so that users could stream with the best performance possible without having to implement difficult algorithms. DAQmx does not provide documentation for its raw data format.

flatzo commented 11 years ago

Thanks for your prompt response.

flatzo commented 11 years ago

This might be useful if planning on supporting it someday :

http://digital.ni.com/public.nsf/allkb/0FAD8D1DC10142FB482570DE00334AFB?OpenDocument