ctest 93% test passed, 3 test failed

[bytesByHarsh]

While running

     ctest --build-config Release

I am getting error

93% tests passed, 3 tests failed out of 40

Total Test time (real) = 107.73 sec

The following tests FAILED:
         20 - darknet_importer_test (Failed)
         39 - wrap-test (Failed)
         40 - ell-python-interface-test (Failed)
Errors while running CTest

For detailed error I addded -VV in above commnd

Got the following error:

===================================================================
20: ERROR: darknet_to_ell_importer_test (unittest.loader._FailedTest)
20: ----------------------------------------------------------------------
20: ImportError: Failed to import test module: darknet_to_ell_importer_test
20: Traceback (most recent call last):
20:   File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 14, in swig_import_helper
20:     return importlib.import_module(mname)
20:   File "C:\<DIR>\Miniconda3\envs\py36\lib\importlib\__init__.py", line 126, in import_module
20:     return _bootstrap._gcd_import(name[level:], package, level)
20:   File "<frozen importlib._bootstrap>", line 994, in _gcd_import
20:   File "<frozen importlib._bootstrap>", line 971, in _find_and_load
20:   File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
20: ModuleNotFoundError: No module named 'ell._ell_py'
20:
20: During handling of the above exception, another exception occurred:
20:
20: Traceback (most recent call last):
20:   File "C:\<DIR>\Miniconda3\envs\py36\lib\unittest\loader.py", line 153, in loadTestsFromName
20:     module = __import__(module_name)
20:   File "E:\Git\ELL\build\tools\importers\darknet\test\darknet_to_ell_importer_test.py", line 30, in <module>
20:     import ell
20:   File "E:\Git\ELL\build\interfaces\python\package\ell\__init__.py", line 42, in <module>
20:     from . import data
20:   File "E:\Git\ELL\build\interfaces\python\package\ell\data\__init__.py", line 9, in <module>
20:     from ..ell_py import AutoDataVector, \
20:   File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 17, in <module>
20:     _ell_py = swig_import_helper()
20:   File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 16, in swig_import_helper
20:     return importlib.import_module('_ell_py')
20:   File "C:\<DIR>\Miniconda3\envs\py36\lib\importlib\__init__.py", line 126, in import_module
20:     return _bootstrap._gcd_import(name[level:], package, level)
20: ImportError: Module use of python37.dll conflicts with this version of Python.
20:
20:
20: ----------------------------------------------------------------------
20: Ran 1 test in 0.000s
20:
20: FAILED (errors=1)
20/40 Test #20: darknet_importer_test ............***Failed    0.63 sec

The below error is received which I am not able to understand that why it is coming as numpy is already installed in miniconda and I am able to import numpy as np normally in same cmd window after running python in it.

test 39
      Start 39: wrap-test

39: Test command: C:\<DIR>\Miniconda3\python.exe "test.py"
39: Test timeout computed to be: 10000000
39: Traceback (most recent call last):
39:   File "test.py", line 36, in <module>
39:     raise err
39:   File "test.py", line 25, in <module>
39:     import ell
39:   File "E:\Git\ELL\build\interfaces\python\package\ell\__init__.py", line 42, in <module>
39:     from . import data
39:   File "E:\Git\ELL\build\interfaces\python\package\ell\data\__init__.py", line 9, in <module>
39:     from ..ell_py import AutoDataVector, \
39:   File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 334, in <module>
39:     import numpy as np
39: ModuleNotFoundError: No module named 'numpy'
39/40 Test #39: wrap-test ........................***Failed    0.16 sec
test 40
      Start 40: ell-python-interface-test

40: Test command: C:\<DIR>\Miniconda3\python.exe "test.py"
40: Test timeout computed to be: 10000000
40: Traceback (most recent call last):
40:   File "test.py", line 45, in <module>
40:     raise err
40:   File "test.py", line 10, in <module>
40:     import ell
40:   File "E:\Git\ELL\build\interfaces\python\package\ell\__init__.py", line 42, in <module>
40:     from . import data
40:   File "E:\Git\ELL\build\interfaces\python\package\ell\data\__init__.py", line 9, in <module>
40:     from ..ell_py import AutoDataVector, \
40:   File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 334, in <module>
40:     import numpy as np
40: ModuleNotFoundError: No module named 'numpy'
40/40 Test #40: ell-python-interface-test ........***Failed    0.15 sec

[lovettchris]

did you forget to activate your miniconda environment and run ctest from there? You also need to run the initial cmake and build from your activated your miniconda environment otherwise it will not build the ell_py.pyd python module. If you did not run cmake from your miniconda environment you will need to delete your build folder and start over. cmake won't be able to change your build from non-python to python.

[bytesByHarsh]

I had activated the miniconda environment and also the initial cmake and build were run.

[lovettchris]

Oh, one more thing, you need to make sure openblas is in your PATH environment, assuming you have a haswell CPU try this:

set PATH=%PATH%;E:\git\ELL\external\OpenBLASLibs.0.2.19.3\build\native\x64\haswell\bin;

For more info see "Path Environment" section of INSTALL-Windows.md

[bytesByHarsh]

Yes I had done this also. And also included this path in PATH variable in properties of computer->advance settings->Environment Variables and then adding this address in PATH variable

[lovettchris]

can you show me the results of "dir E:\Git\ELL\build\interfaces\python\package\ell\release" ? You should see an _ell_py.pyd module there, and if you run the http://www.dependencywalker.com/ on this module you might find what is missing (ignore the errors reported about anything named API-MS-WIN-*, but if you see anything else missing then that will be it. I find the "View/Collapse All" option is handy. This is what I see on my machine:

[bytesByHarsh]

Also I am using Intel(R) Core(TM) i5-8300H CPU @2.3GHz .

[lovettchris]

Interesting. I wonder then if you have a CPU that our OpenBlas doesn't support... Do you see output like this from cmake?

-- Processor family: 6, model: 58 -- Using OpenBLAS compiled for sandybridge -- Using BLAS include path: D:/git/ELL/ELL/external/OpenBLASLibs.0.2.19.3/build/native/x64/sandybridge/include -- Using BLAS library: D:/git/ELL/ELL/external/OpenBLASLibs.0.2.19.3/build/native/x64/sandybridge/lib/libopenblas.dll.a -- Using BLAS DLLs: libopenblas.dll;libgcc_s_seh-1.dll;libgfortran-3.dll;libquadmath-0.dll

[bytesByHarsh]

When I run "cmake ." Following is a part of message that is shown

-- Processor family: 6, model: 158 -- Using OpenBLAS compiled for haswell -- Using BLAS include path: E:/Git/ELL/external/OpenBLASLibs.0.2.19.3/build/native/x64/haswell/include -- Using BLAS library: E:/Git/ELL/external/OpenBLASLibs.0.2.19.3/build/native/x64/haswell/lib/libopenblas.dll.a -- Using BLAS DLLs: libopenblas.dll;libgcc_s_seh-1.dll;libgfortran-3.dll;libquadmath-0.dll -- Found LLVM 6.0.1 -- Using LLVMConfig.cmake in: E:/Git/ELL/external/LLVMNativeWindowsLibs.x64.6.0.1/llvm-6.0/lib/cmake/llvm

[lovettchris]

Ok great, so OpenBlas should work, and depends.exe is finding your OpenBlas so all should be good. I also noticed you reported error "ModuleNotFoundError: No module named 'numpy'" -- so did you install OpenCV in your python environment? Installing OpenCV should also bring in numpy...

[bytesByHarsh]

Yes opencv was installed it. I tried installing numpy using pip it showed that numpy was already installed, so it tried

conda install -c conda-forge numpy

still no use, same error is showing.

[bytesByHarsh]

Just like I said earlier if type following in cmd line no error is shown:

python import numpy as np

I am able import numpy that means numpy is installed but while running the test it is showing this error

[lovettchris]

Sorry I missed your earlier comment about being able to import numpy, this is now really weird. You can import numpy, so why can't ell. Can you try this:

cd E:\Git\ELL\build\interfaces\python\package
python
import ell

If this fails with the same error then I may need to ask you to debug into this using VS code.

[bytesByHarsh]

import ell

Traceback (most recent call last): File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 14, in swig_import_helper return importlib.import_module(mname) File "C:\Users\harsh\Miniconda3\envs\py36\lib\importlib__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'ell._ell_py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "E:\Git\ELL\build\interfaces\python\package\ell__init.py", line 42, in from . import data File "E:\Git\ELL\build\interfaces\python\package\ell\data\init.py", line 9, in from ..ell_py import AutoDataVector, \ File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 17, in _ell_py = swig_import_helper() File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 16, in swig_import_helper return importlib.import_module('_ell_py') File "C:\Users\harsh\Miniconda3\envs\py36\lib\importlib\init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: Module use of python37.dll conflicts with this version of Python.

[lovettchris]

Ah this is a clue:

ImportError: Module use of python37.dll conflicts with this version of Python.

it is odd this is mentioning python37 when your environment is "C:\Users\harsh\Miniconda3\envs\py36". Did you do a system wide install of python37? Perhaps it is finding that instead of your python36...

What do you get from "where python" ?

[bytesByHarsh]

It is returning:

C:\Users\harsh\Miniconda3\python.exe C:\Users\harsh\AppData\Local\Programs\Python\Python36\python.exe

Oh yes Python 3.7.1 is installed in my system

<DIR>python
Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.

After activating py36:

(py36) E:\Git\ELL\build\interfaces\python\package>python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 18:50:55) [MSC v.1915 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

[lovettchris]

so "C:\Users\harsh\Miniconda3\python.exe" is a python 3.7 then eh? Can you remove C:\Users\harsh\Miniconda3\ from your PATH environment, and activate py36, and run the ELL rebuild.cmd ? I suspect something in there got confused about the presence of python 3.7 in your PATH...

[bytesByHarsh]

After removing it from the path, following error message is shown after "import ell"

Traceback (most recent call last): File "", line 1, in File "E:\Git\ELL\build\interfaces\python\package\ell__init.py", line 42, in from . import data File "E:\Git\ELL\build\interfaces\python\package\ell\data\init.py", line 9, in from ..ell_py import AutoDataVector, \ File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 17, in _ell_py = swig_import_helper() File "E:\Git\ELL\build\interfaces\python\package\ell\ell_py.py", line 16, in swig_import_helper return importlib.import_module('_ell_py') File "C:\Users\harsh\Miniconda3\envs\py36\lib\importlib\init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: DLL load failed: The specified module could not be found.

[lovettchris]

Assuming "depends.exe" is still happy with _ell_py.pyd, this error is the one that normally means a DLL cannot be found, in debugging "import ell" in vs code if you break on all exceptions you might get more information.

[bytesByHarsh]

So I deleted the build folder and started again and with response to my PR #189 changes have been made for the ctest command for windows as well and now 100% tests are passed. Still really not sure what caused the problem in the first place but now the problem is solved

Thanks for the help @lovettchris

[lovettchris]

Cool, thanks for the update.

[jesuspicazo]

Hi, I'm sorry for reopening this thread again but I'm having the exact same issue than harshmittal2210 but in Linux 18.04. Everything is point by point exactly the same as in the description he does of the problem. Have you released any solution for the ctests similar as you did for the windows version?

Thanks,

Jesus