Closed mtpalmer closed 5 years ago
Hi, I guess mdfreader version is 2.7.7 and not 2.2.7 ? Did you correctly build dataRead on your platform ? You can check with python setup.py build_ext. If you have difficulties, I could try to make a wheel for 32bit windows (already existing for 64bit). The warning you have can be filtered if you wish. It just informs that mdfreader is going to a fallback code in python instead of cython dataRead module, therefore slower. For the error regarding the string, there is indeed a bug there in _bits_to_bytes(), thanks for this. I will correct it in a new dev branch.
Hi,
You are correct, typo there, 2.7.7.
Thank you for reading through my issues, glad I could help with finding a bug!
I cloned the master repo and attempted to rebuild with setup.py.
I am having issues building dataRead. For some reason, it is failing on the cython library import in the setup script. It can't seem to find Cython.setuptools and I don't seem to have a setuptools folder/whatever its called in my Cython library. I re-built the cython library without errors and I still don't have it.
Current Version of cython: 0.26
In the setup.py script, I added a print function for failure to import Cython.setuptools.build_ext on line 12
print "Failure to import"
I additionally changed the Cython import to
from Cython.Distutils import build_ext
I do have the standalone visual c++ 9.0 installed, but I get the following error:
mdfreader-master>python setup.py build_ext --inplace
running build_ext
building 'dataRead' extension
Traceback (most recent call last):
File "setup.py", line 133, in
I additionally attempted to use the --compiler flag
python setup.py build_ext --compiler=msvc --inplace and it failed with the same error
I additionally made the following changes to the attempt to build with cython, starting on lines 125, following a cython build tutorial.
try: # with cython installed from Cython.Build import cythonize import numpy ext_modules = cythonize("dataRead.pyx", include_path = [numpy.get_include()]) setup(name=name, version=version, description=description, long_description=long_description, url=url, author=author, author_email=author_email, license=license, classifiers=classifiers, keywords=keywords, packages=packages, install_requires=install_requires, extras_require=extras_require, entry_points=entry_points, ext_modules=ext_modules) except: # without cython import traceback
traceback.print_exc()
It actually tries to build the .pyx but comes back with the following errors:
I, then, run the setup.py cmd with the following command
python setup.py install
And get the following Cython Compile Error and indication that I could not import cython.setuptools:
dataRead.pyx:332:37: Attempting to index non-array type 'unsigned int'
dataRead.pyx:332:56: Attempting to index non-array type 'unsigned int'
dataRead.pyx:332:75: Attempting to index non-array type 'unsigned int'
dataRead.pyx:332:93: Attempting to index non-array type 'unsigned int'
dataRead.pyx:406:37: Attempting to index non-array type 'int'
dataRead.pyx:406:56: Attempting to index non-array type 'int'
dataRead.pyx:406:75: Attempting to index non-array type 'int'
dataRead.pyx:406:93: Attempting to index non-array type 'int'
dataRead.pyx:482:37: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:482:56: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:482:75: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:482:94: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:483:37: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:483:56: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:483:75: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:483:93: Attempting to index non-array type 'unsigned long long'
dataRead.pyx:602:37: Attempting to index non-array type 'long long'
dataRead.pyx:602:56: Attempting to index non-array type 'long long'
dataRead.pyx:602:75: Attempting to index non-array type 'long long'
dataRead.pyx:602:94: Attempting to index non-array type 'long long'
dataRead.pyx:603:37: Attempting to index non-array type 'long long'
dataRead.pyx:603:56: Attempting to index non-array type 'long long'
dataRead.pyx:603:75: Attempting to index non-array type 'long long'
dataRead.pyx:603:93: Attempting to index non-array type 'long long'
Traceback (most recent call last):
File "setup.py", line 127, in
Hi, Dev branch should arrange your situation there, both for compilation and string bug. However, there are still warnings from compilation I cannot figure out yet (very rare case I think of byte swapping with long long type + numpy API deprecation) Please confirm if it solves your issue.
Corrected issue with string bug.
Have an error with compilation where the include directory for /site-packages/numpy/core is expected to be in the distutils config.
Hi, Thanks for the confirmation. Those compilation warning cannot be avoided as it is related to bit shift.
Python version
Platform information
Numpy version
Please write here the output of printing
numpy.__version__
1.14.0mdfreader version
Please write here the output of printing
mdfreader.__version__
2.2.7Description
MDF Version 3.20
Summary :
Attempt to fully open file does not work. Opened file without reading and read the data of each channel to see which channel(s) were causing the issue. I end up receiving several warnings for ImportError on dataRead and one fatal exception when trying to read a string measurement channel.
Troubleshooting:
Opening the file with
mdfobj = mdfreader.mdf('.mdf')
C:\Python27\lib\site-packages\mdfreader\mdf3reader.py:553: UserWarning: Unexpected error: (<type 'exceptions.ImportError'>, ImportError('No module named dataRead',), <traceback object at 0x116B5C10>) warn('Unexpected error: {}'.format(exc_info())) C:\Python27\lib\site-packages\mdfreader\mdf3reader.py:554: UserWarning: dataRead crashed, back to python data reading warn('dataRead crashed, back to python data reading') Traceback (most recent call last): File "C:\cat\cml_pi_analyze\src\read_data_mdf.py", line 31, in
mdfobj = mdfreader.mdf('..\data\Drill_Air78.mdf')
File "C:\Python27\lib\site-packages\mdfreader\mdf.py", line 159, in init
compression=compression)
File "C:\Python27\lib\site-packages\mdfreader\mdfreader.py", line 391, in read
convertAfterRead, filterChannelNames, compression)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 922, in read3
buf.read(channelSet, self.fileName) # reads datablock potentially containing several channel groups
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 714, in read
self.loadSorted(self[recordID]['record'], nameList=channelSet)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 738, in loadSorted
return record.readSortedRecord(self.fid, self.pointerToData, nameList)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 561, in readSortedRecord
channel.posByteEnd])[0]
struct.error: unpack requires a string argument of length 7
However, opening with the following script reveals one string channel having issues, with a bunch of import warnings on most data channels.
As an example of the warnings:
t_7 {'description': u'', 'masterType': 1, 'master': 't_7', 'data': None, 'id': (7, 0, 0), 'unit': u's'} C:\Python27\lib\site-packages\mdfreader\mdf3reader.py:553: UserWarning: Unexpected error: (<type 'exceptions.ImportError'>, ImportError('No module named dataRead',), <traceback object at 0x1176A170>) warn('Unexpected error: {}'.format(exc_info())) [0.05094666]
I then actually crashes on, which is a string measurement.
software_group_date_code_str {'description': u'', 'masterType': 1, 'master': 't_7', 'data': None, 'id': (7, 0, 1), 'unit': ''} C:\Python27\lib\site-packages\mdfreader\mdf3reader.py:553: UserWarning: Unexpected error: (<type 'exceptions.ImportError'>, ImportError('No module named dataRead',), <traceback object at 0x116FEB48>) warn('Unexpected error: {}'.format(exc_info())) Traceback (most recent call last): File "C:\cat\cml_pi_analyze\src\read_data_mdf.py", line 38, in
print yop.getChannelData(key)
File "C:\Python27\lib\site-packages\mdfreader\mdfreader.py", line 458, in getChannelData
vect = self._getChannelData3(channelName, raw_data)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 1006, in _getChannelData3
self.read3(fileName=None, info=self.info, channelList=[channelName], convertAfterRead=False)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 922, in read3
buf.read(channelSet, self.fileName) # reads datablock potentially containing several channel groups
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 714, in read
self.loadSorted(self[recordID]['record'], nameList=channelSet)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 738, in loadSorted
return record.readSortedRecord(self.fid, self.pointerToData, nameList)
File "C:\Python27\lib\site-packages\mdfreader\mdf3reader.py", line 561, in readSortedRecord
channel.posByteEnd])[0]
error: unpack requires a string argument of length 7
unpack requires a string argument of length 7
Total size of mdf file: 3.2 MB
mdfvalidator