search

silx-kit / pyFAI

Fast Azimuthal Integration in Python
Other
106 stars 95 forks source link

Unit test problems #75

Closed picca closed 11 years ago

picca commented 11 years ago

Hello Jerome,

I played a little bit with the unittest when I saw, plenty of

Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Singular determinant, Hessian undefined Singular determinant, Hessian undefined Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Singular determinant, Hessian undefined Failed to find root using second order expansion

So I tryed to find the source of these messages.

1) first it is not easy to find which test is causing theses messages.

Now I executes this line at the test failed. picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v testAzimuthalIntegrator WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU test_cythonSP_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_cython_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_cython_vs_numpy (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_numpy_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_splitBBox (testAzimuthalIntegrator.test_flatimage) ... ok test_splitPixel (testAzimuthalIntegrator.test_flatimage) ... ok test_mask (testAzimuthalIntegrator.test_saxs) test the generation of mask ... /home/picca/Projets/pyFAI/test/testimages/bsa_013_01.edf FAIL test_dark (testAzimuthalIntegrator.test_setter) ... ok test_flat (testAzimuthalIntegrator.test_setter) ... ok

FAIL: test_mask (testAzimuthalIntegrator.test_saxs)

test the generation of mask

Traceback (most recent call last): File "testAzimuthalIntegrator.py", line 253, in test_mask assert abs(self.ai.makeMask(data, mask=mask, dummy= -2, delta_dummy=1.1).astype(int) - fabio.open(self.maskDummy).data).max() == 0 AssertionError

Ran 9 tests in 153.342s

FAILED (failures=1)

this is very strange since a simple PYTHONPATH=... python setup.py test does not failed like this.

Is there some non-ran tests with

python setup.py test ?

Cheers

Fred

picca commented 11 years ago

picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v testGeometry WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU testGeometryFunctions (testGeometry.TestGeometry) ... ERROR testSolidAngle (testGeometry.TestSolidAngle) ... ok

ERROR: testGeometryFunctions (testGeometry.TestGeometry)

Traceback (most recent call last): File "testGeometry.py", line 96, in testGeometryFunctions func, statargs, varargs, kwds, expectedFail = self.param TypeError: 'NoneType' object is not iterable

Ran 2 tests in 15.544s

FAILED (errors=1)

picca commented 11 years ago

picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v test_ocl_lut.py WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/lib/python2.7/unittest/main.py", line 12, in main(module=None) File "/usr/lib/python2.7/unittest/main.py", line 94, in init self.parseArgs(argv) File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs self.createTests() File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests self.module) File "/usr/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = import('.'.join(parts_copy)) File "test_ocl_lut.py", line 4, in import fabio, pyopencl ImportError: No module named pyopencl

picca commented 11 years ago

picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v testOpenCL WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:testOpenCL:OpenCL module (pyopencl) is not present, skip tests. No module named pyopencl. test_OpenCL (testOpenCL.test_mask) ... ERROR test_OpenCL_LUT (testOpenCL.test_mask) ... ERROR

ERROR: test_OpenCL (testOpenCL.test_mask)

Traceback (most recent call last): File "testOpenCL.py", line 80, in test_OpenCL ids = ocl.select_device(devtype, extensions=["cl_khr_int64_base_atomics"]) AttributeError: 'NoneType' object has no attribute 'select_device'

ERROR: test_OpenCL_LUT (testOpenCL.test_mask)

Traceback (most recent call last): File "testOpenCL.py", line 103, in test_OpenCL_LUT ids = ocl.select_device(devtype, best=True) AttributeError: 'NoneType' object has no attribute 'select_device'

Ran 2 tests in 25.459s

FAILED (errors=2)

picca commented 11 years ago

picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v testPeakPicking WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU test_peakPicking (testPeakPicking.test_peak_picking) first test peak-picking then checks the geometry found is OK ... Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion Failed to find root using second order expansion

kif commented 11 years ago

Hi Fred,

Singular determinant, Hessian undefined Failed to find root using second order expansion

So I tryed to find the source of these messages.

This is in the bilinear (binary) module: I implemented a peak-picking with second order expansion for sub-pixel refinement. It was less than a week ago. For know it falls back on the previous method in case of error but this will change once the "correct" way is defined.

1) first it is not easy to find which test is causing theses messages.

Now I executes this line at the test failed. picca@mordor:~/Projets/pyFAI/test$ PYTHONPATH=../build/lib.linux-i686-2.7/ python -m unittest -v testAzimuthalIntegrator WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU WARNING:root:Exception No module named fftw3: FFTw3 not available. Falling back on Scipy ERROR:pyFAI.opencl:Unable to import pyOpenCl. Please install it from: http://pypi.python.org/pypi/pyopencl ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim ERROR:pyFAI.azimuthalIntegrator:Unable to import pyFAI.ocl_azim_lut for Look-up table based azimuthal integration on GPU test_cythonSP_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_cython_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_cython_vs_numpy (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_numpy_vs_fit2d (testAzimuthalIntegrator.test_azim_halfFrelon) ... ok test_splitBBox (testAzimuthalIntegrator.test_flatimage) ... ok test_splitPixel (testAzimuthalIntegrator.test_flatimage) ... ok test_mask (testAzimuthalIntegrator.test_saxs) test the generation of mask ... /home/picca/Projets/pyFAI/test/testimages/bsa_013_01.edf FAIL test_dark (testAzimuthalIntegrator.test_setter) ... ok test_flat (testAzimuthalIntegrator.test_setter) ... ok

FAIL: test_mask (testAzimuthalIntegrator.test_saxs)

test the generation of mask

Traceback (most recent call last): File "testAzimuthalIntegrator.py", line 253, in test_mask assert abs(self.ai.makeMask(data, mask=mask, dummy= -2, delta_dummy=1.1).astype(int) - fabio.open(self.maskDummy).data).max() == 0 AssertionError

Ran 9 tests in 153.342s

FAILED (failures=1)

this is very strange since a simple PYTHONPATH=... python setup.py test does not failed like this.

Is there some non-ran tests with

python setup.py test ?

Could be ... look at the test_all script where the test suite is defined. I am on Week-end and while I have a network connection, I have no development tools with me.

Cheers,

Jerome Kieffer jerome.kieffer@terre-adelie.org

kif commented 11 years ago

The problem comes from the way of launching the tests: do not force load the unittest module. python -m unittest -v

Instead there is a test suite "test_all.py" that agregate all tests.

Some tests cannot be run using automatic test discovery because of test generation (i.e. testGeometry) to create a lot of tests with different configuration.