search

SyneRBI / SIRF-SuperBuild

SIRF CMake SuperBuild
http://www.ccpsynerbi.ac.uk
Apache License 2.0
15 stars 18 forks source link

Windows: Build success but python cannot load SIRF #729

Open paskino opened 2 years ago

paskino commented 2 years ago

I built locally on Windows10 with VS community edition and NVIDIA SDK.

conda create --name sirf_b2 python numpy openblas matplotlib=3.4 docopt cmake deprecation -c conda-forge
conda activate sirf_b2
cmake ../SIRF-SuperBuild -DSIRF_TAG=origin/master -DSTIR_TAG=origin/master -DCIL_TAG=origin/master -DBUILD_Gadgetron=OFF -DUSE_SYSTEM_HDF5=OFF -DUSE_SYSTEM_Boost=ON -DBoost_ROOT=C:\Apps\boost_1_72_0 -DUSE_SYSTEM_FFTW=OFF

After sourcing the env_sirf.PS1 environment file on PowerShell, I can run SIRF C++ unit tests (excluding the MR tests as I haven't built Gadgetron)

cd builds\SIRF\build
ctest -v --output-on-failure -R CPLUS -E MR -C Release

which outputs:

Test project C:/Users/ofn77899/Dev/buildVC/builds/SIRF/build
    Start  6: REG_TEST_CPLUSPLUS
1/7 Test  #6: REG_TEST_CPLUSPLUS ......................   Passed    4.40 sec
    Start  9: PET_TESTS_CPLUSPLUS_1
2/7 Test  #9: PET_TESTS_CPLUSPLUS_1 ...................   Passed   66.29 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
3/7 Test #10: PET_TESTS_CPLUSPLUS_4 ...................   Passed   65.26 sec
    Start 14: SYN_TEST_STIR_TO_NIFTI_CPLUSPLUS
4/7 Test #14: SYN_TEST_STIR_TO_NIFTI_CPLUSPLUS ........   Passed    1.05 sec
    Start 15: SYN_TEST_GADGETRON_TO_NIFTI_CPLUSPLUS
5/7 Test #15: SYN_TEST_GADGETRON_TO_NIFTI_CPLUSPLUS ...   Passed    0.53 sec
    Start 18: SYN_TEST_CPLUSPLUS
6/7 Test #18: SYN_TEST_CPLUSPLUS ......................***Failed   12.42 sec
// ----------------------------------------------------------------------- //
//                  Starting STIRImageData->NiftiImageData test...
//------------------------------------------------------------------------ //

INFO: Reading C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/zenodo/dicom_as_nifti.nii via ITK non-DICOM IO

WARNING: Unable to determine patient position. Internally this will generally be handled by assuming HFS

Saving image to file (results/stir_to_nifti.nii)...done.

Image comparison: metadata do not match, doing resampling...

No transformations set, using identity.
// ----------------------------------------------------------------------- //
//                  Finished STIRImageData->NiftiImageData test.
//------------------------------------------------------------------------ //
// ----------------------------------------------------------------------- //
//                  Starting GadgetronImageData->NiftiImageData test...
//------------------------------------------------------------------------ //
group 2019-12-11 16:38:14
variable image_1
number of images: 31
image data type: 5
variable xml

GadgetronImagesVector::set_up_geom_info(). Warning, you set up geometry for slices whose width is not their distance.This setup does probably not account for overlaps or gaps between slices.

Image comparison: metadata do not match, doing resampling...

No transformations set, using identity.
// ----------------------------------------------------------------------- //
//                  Finished GadgetronImageData->NiftiImageData test.
//------------------------------------------------------------------------ //
// ----------------------------------------------------------------------- //
//                  Starting complex resampler test...
//------------------------------------------------------------------------ //
Started reading acquisitions from C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/grappa2_1rep.h5
0%..10%..20%..30%..40%..49%..59%..69%..79%..89%..98%..
Finished reading acquisitions from C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/grappa2_1rep.h5
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed

Here's the error:
        Server running Gadgetron not accessible

    Start 20: SIRF_TEST_COMMON_CPLUSPLUS
7/7 Test #20: SIRF_TEST_COMMON_CPLUSPLUS ..............   Passed    0.10 sec

86% tests passed, 1 tests failed out of 7

Total Test time (real) = 150.22 sec

The following tests FAILED:
         18 - SYN_TEST_CPLUSPLUS (Failed)
Errors while running CTest

All unit tests involving python ctest -v --output-on-failure -R PYTHON -E MR -C Release fail with

File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

All output here

Test project C:/Users/ofn77899/Dev/buildVC/builds/SIRF/build
    Start  7: REG_TESTS_PYTHON
1/8 Test  #7: REG_TESTS_PYTHON .....................***Failed    0.09 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose

    Start  8: REG_PYTHON_ALGEBRA
2/8 Test  #8: REG_PYTHON_ALGEBRA ...................***Failed    1.55 sec
E
======================================================================
ERROR: test_algebra (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_algebra
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Registration\pReg\tests\test_algebra.py", line 22, in <module>
    import sirf.Reg as reg
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\Reg.py", line 28, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 11: PET_TESTS_PYTHON
3/8 Test #11: PET_TESTS_PYTHON .....................***Failed    0.10 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose

    Start 12: PET_PYTHON_ALGEBRA
4/8 Test #12: PET_PYTHON_ALGEBRA ...................***Failed    1.62 sec
E
======================================================================
ERROR: test_algebra (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_algebra
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\xSTIR\pSTIR\tests\test_algebra.py", line 22, in <module>
    import sirf.STIR as pet
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\STIR.py", line 39, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 13: PET_PYTHON_OBJFUN
5/8 Test #13: PET_PYTHON_OBJFUN ....................***Failed    1.48 sec
E
======================================================================
ERROR: test_ObjectiveFunction (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_ObjectiveFunction
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\xSTIR\pSTIR\tests\test_ObjectiveFunction.py", line 22, in <module>
    import sirf.STIR as pet
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\STIR.py", line 39, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 16: SYN_TEST_GADGETRON_TO_NIFTI_PYTHON
6/8 Test #16: SYN_TEST_GADGETRON_TO_NIFTI_PYTHON ...***Failed    1.49 sec
Traceback (most recent call last):
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Synergistic\pSyn\tests\tests_syn.py", line 3, in <module>
    from sirf.SIRF import ImageData
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

    Start 17: SYN_TEST_STIR_TO_NIFTI_PYTHON
7/8 Test #17: SYN_TEST_STIR_TO_NIFTI_PYTHON ........***Failed    1.48 sec
Traceback (most recent call last):
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Synergistic\pSyn\tests\tests_syn.py", line 3, in <module>
    from sirf.SIRF import ImageData
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

    Start 19: SYN_TEST_PYTHON
8/8 Test #19: SYN_TEST_PYTHON ......................***Failed    0.12 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose

0% tests passed, 8 tests failed out of 8

Total Test time (real) =   8.16 sec

The following tests FAILED:
          7 - REG_TESTS_PYTHON (Failed)
          8 - REG_PYTHON_ALGEBRA (Failed)
         11 - PET_TESTS_PYTHON (Failed)
         12 - PET_PYTHON_ALGEBRA (Failed)
         13 - PET_PYTHON_OBJFUN (Failed)
         16 - SYN_TEST_GADGETRON_TO_NIFTI_PYTHON (Failed)
         17 - SYN_TEST_STIR_TO_NIFTI_PYTHON (Failed)
         19 - SYN_TEST_PYTHON (Failed)
Errors while running CTest
paskino commented 2 years ago

The environment variables PATH and PYTHONPATH seem to be correct.

PATH

My $Env:Path is 

C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Apps\miniconda3\envs\sirf_b2;C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin;C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin;C:\Apps\miniconda3\envs\sirf_b2\Library\bin;C:\Apps\miniconda3\envs\sirf_b2\Scripts;C:\Apps\miniconda3\envs\sirf_b2\bin;C:\Apps\miniconda3\condabin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\ProgramData\chocolatey\bin;C:\WINDOWS\System32\OpenSSH;C:\Program Files\Docker\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Apps\Vagrant\bin;C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps;C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\ofn77899\AppData\Local\Programs\Git\cmd;C:\Apps\GitHub CLI;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2;C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2

PYTHONPATH

C:\Users\ofn77899\Dev\buildVC\INSTALL\python;C:\Users\ofn77899\Dev\buildVC\INSTALL\python;C:\Users\ofn77899\Dev\buildVC\INSTALL\python;
evgueni-ovtchinnikov commented 2 years ago

@paskino You probably need to add C:\Users\ofn77899\Dev\buildVC\INSTALL\python to PATH to help Python find _pysirf.pyd and other SIRF Python dll's.

evgueni-ovtchinnikov commented 2 years ago

@paskino regarding MR (and some SYN) tests: can you not run Gadgetron on SIRF VM?

paskino commented 2 years ago

Thanks @evgueni-ovtchinnikov , I tried to add to PATH with 

$Env:Path="C:\Users\ofn77899\Dev\buildVC\INSTALL\python;$Env:Path"
(sirf_b2) PS C:\Users\ofn77899\Dev\buildVC\builds\SIRF\build> $Env:Path                                                 C:\Users\ofn77899\Dev\buildVC\INSTALL\python;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Apps\miniconda3\envs\sirf_b2;
C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin;
C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin;
C:\Apps\miniconda3\envs\sirf_b2\Library\bin;
C:\Apps\miniconda3\envs\sirf_b2\Scripts;
C:\Apps\miniconda3\envs\sirf_b2\bin;
C:\Apps\miniconda3\condabin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;
C:\WINDOWS\system32;C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\WINDOWS\System32\WindowsPowerShell\v1.0;
C:\ProgramData\chocolatey\bin;
C:\WINDOWS\System32\OpenSSH;
C:\Program Files\Docker\Docker\resources\bin;
C:\ProgramData\DockerDesktop\version-bin;
C:\Apps\Vagrant\bin;
C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps;
C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin;
C:\Users\ofn77899\AppData\Local\Programs\Git\cmd;
C:\Apps\GitHub CLI;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2;
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2;
C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps;
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2

but still get the same error.

KrisThielemans commented 2 years ago

I think you have to compare your 2 paths from within python, as it'll change where it finds DLLs compare to where the executable finds them. I still think it's a conflict between the HDF5/FFTW/Boost library used by SIRF and the ones installed by conda and hence used by conda's python.

See also https://bugs.python.org/issue43173 for some info on where it finds DLLs

paskino commented 2 years ago

Just to be clear, I did not install HDF5, FFTW or Boost with conda

conda list
# packages in environment at C:\Apps\miniconda3\envs\sirf_b2:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.6.15            h5b45459_0    conda-forge
certifi                   2022.6.15       py310h5588dad_0    conda-forge
cmake                     3.23.2               h39d44d4_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
deprecation               2.1.0              pyh9f0ad1d_0    conda-forge
docopt                    0.6.2                      py_1    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
gettext                   0.19.8.1          ha2e2712_1008    conda-forge
glib                      2.70.2               h7755175_4    conda-forge
glib-tools                2.70.2               h7755175_4    conda-forge
gst-plugins-base          1.20.3               he07aa86_0    conda-forge
gstreamer                 1.20.3               hdff456e_0    conda-forge
icu                       70.1                 h0e60522_0    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
jpeg                      9e                   h8ffe710_1    conda-forge
kiwisolver                1.4.3           py310h476a331_0    conda-forge
krb5                      1.19.3               h1176d77_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      3.0                  h0e60522_0    conda-forge
libblas                   3.9.0              15_win64_mkl    conda-forge
libcblas                  3.9.0              15_win64_mkl    conda-forge
libclang                  14.0.5          default_h77d9078_0    conda-forge
libclang13                14.0.5          default_h77d9078_0    conda-forge
libdeflate                1.12                 h8ffe710_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libflang                  5.0.0           h6538335_20180525    conda-forge
libglib                   2.70.2               h3be07f2_4    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
liblapack                 3.9.0              15_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libopenblas               0.3.20          pthreads_hc469a61_0    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libtiff                   4.4.0                h2ed3b44_1    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.2.2                h57928b3_0    conda-forge
libwebp-base              1.2.2                h8ffe710_1    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.12               h8ffe710_1    conda-forge
llvm-meta                 5.0.0                         0    conda-forge
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
matplotlib                3.4.3           py310h5588dad_2    conda-forge
matplotlib-base           3.4.3           py310h79a7439_2    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
nose                      1.3.7                   py_1006    conda-forge
numpy                     1.22.4          py310hed7ac4c_0    conda-forge
openblas                  0.3.20          pthreads_h543f93c_0    conda-forge
openjpeg                  2.4.0                hb211442_1    conda-forge
openmp                    5.0.0                    vc14_1    conda-forge
openssl                   1.1.1p               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pcre                      8.45                 h0e60522_0    conda-forge
pillow                    9.1.1           py310h767b3fd_1    conda-forge
pip                       22.1.2             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.4          py310hbabf5d4_1    conda-forge
pyqt5-sip                 12.9.0          py310h8a704f9_1    conda-forge
python                    3.10.5          h9a09f29_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
qt-main                   5.15.4               h467ea89_2    conda-forge
setuptools                62.6.0          py310h5588dad_0    conda-forge
sip                       6.5.1           py310h8a704f9_2    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.38.5               h8ffe710_0    conda-forge
tbb                       2021.5.0             h2d74725_1    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1             py310he2412df_3    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
zlib                      1.2.12               h8ffe710_1    conda-forge
zstd                      1.5.2                h6255e5f_1    conda-forge
paskino commented 2 years ago 
>>> import os
>>> for el in os.environ['PATH'].split(';'):
...     print (el)
...
C:\Users\ofn77899\Dev\buildVC\INSTALL\python
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Apps\miniconda3\envs\sirf_b2
C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin
C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin
C:\Apps\miniconda3\envs\sirf_b2\Library\bin
C:\Apps\miniconda3\envs\sirf_b2\Scripts
C:\Apps\miniconda3\envs\sirf_b2\bin
C:\Apps\miniconda3\condabin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp
C:\WINDOWS\system32
C:\WINDOWS
C:\WINDOWS\System32\Wbem
C:\WINDOWS\System32\WindowsPowerShell\v1.0
C:\ProgramData\chocolatey\bin
C:\WINDOWS\System32\OpenSSH
C:\Program Files\Docker\Docker\resources\bin
C:\ProgramData\DockerDesktop\version-bin
C:\Apps\Vagrant\bin
C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps
C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin
C:\Users\ofn77899\AppData\Local\Programs\Git\cmd
C:\Apps\GitHub CLI
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2
C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2
paskino commented 2 years ago

I made some progress, installing SIRF directly in the conda environment!

conda create --name sirf_b2 python numpy openblas matplotlib=3.4 docopt cmake deprecation nose -c conda-forge

cmake ../SIRF-SuperBuild -DSIRF_TAG=origin/master -DSTIR_TAG=origin/master \
  -DCIL_TAG=origin/master -DBUILD_Gadgetron=OFF -DUSE_SYSTEM_HDF5=OFF \
  -DUSE_SYSTEM_Boost=ON -DBoost_ROOT=C:\Apps\boost_1_72_0 \
  -DUSE_SYSTEM_FFTW=OFF \
  -DPYTHON_DEST_DIR=%CONDA_PREFIX%\lib\site-packages \
  -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%

Now the unittest fail trying to load a file, although the file in question exists in the appropriate directory.

sirf.Utilities.error: ??? '\'Error opening file "C:\\\\Apps\\\\miniconda3\\\\envs\\\\sirf_b3"\\\\share\\\\SIRF-3.3\\\\data\\\\examples\\\\PET\\\\thorax_single_slice\\\\emission.hv\\n\' exception caught at line 327 of C:\\Users\\ofn77899\\Dev\\buildVC3\\sources\\SIRF\\src\\xSTIR\\cSTIR\\cstir.cpp; the reconstruction engine output may provide more information'
KrisThielemans commented 2 years ago

for el in os.environ['PATH'].split(';'): ... print (el)

and sys.path? (in your previous set-up)

KrisThielemans commented 2 years ago

I made some progress, installing SIRF directly in the conda environment!

great. Are these lines the only difference?

-DPYTHON_DEST_DIR=%CONDA_PREFIX%\lib\site-packages \
  -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%

The file-not-found seems to be due to backslash issues (unless it's really not there). Likely the file-ath is the output of the following

os.path.join(examples_data_path('PET'),'thorax_single_slice','emission.hv')

What's the return value of examples_data_path('PET')? and of your SIRF_PATH environment variable? Content of my env_sirf.PS1 is

# need to be forward slashes, as otherwise cannot find files
$Env:SIRF_PATH = "C:/Users/krisf/Documents/devel/buildVC/SIRF-SuperBuild/sources/SIRF"

configured at https://github.com/SyneRBI/SIRF-SuperBuild/blob/c19d1340dfc623518c3594538cc4a7121a14b670/env_sirf.PS1.in#L7

paskino commented 2 years ago

Looks like that installing in the conda environment works.

. C:\Apps\miniconda3\envs\sirf_b3\bin\env_sirf.PS1

ctest -C Release -R PET_ --output-on-failure             Test project C:/Users/ofn77899/Dev/buildVC3/builds/SIRF/build
    Start  9: PET_TESTS_CPLUSPLUS_1
1/5 Test  #9: PET_TESTS_CPLUSPLUS_1 ............   Passed   53.46 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
2/5 Test #10: PET_TESTS_CPLUSPLUS_4 ............   Passed   54.96 sec
    Start 11: PET_TESTS_PYTHON
3/5 Test #11: PET_TESTS_PYTHON .................   Passed  123.90 sec
    Start 12: PET_PYTHON_ALGEBRA
4/5 Test #12: PET_PYTHON_ALGEBRA ...............   Passed    8.65 sec
    Start 13: PET_PYTHON_OBJFUN
5/5 Test #13: PET_PYTHON_OBJFUN ................   Passed    1.67 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 242.76 sec
KrisThielemans commented 2 years ago

ah, you forgot to source the PS1?

paskino commented 2 years ago

No, I did not use PowerShell, rather command prompt. So I had added the environment variables by hand.

KrisThielemans commented 2 years ago

ok, but what were there values then? Is there a difference in what they should be between PS and CMD?

Also, is the same cause for your original problems (i.e. do yo really have to install in the conda dirs?)

paskino commented 2 years ago

Yes, outside of conda dirs neither PowerShell, nor CMD worked.

I had an issue with setting the path with " on CMD, but now, without double quotes, it seems to be working fine.

set SIRF_PATH=C:/Users/ofn77899/Dev/buildVC3/sources/SIRF
set SIRF_INSTALL_PATH=C:\Apps\miniconda3\envs\sirf_b3
ctest -C Release -R PET_ --output-on-failure
Test project C:/Users/ofn77899/Dev/buildVC3/builds/SIRF/build
    Start  9: PET_TESTS_CPLUSPLUS_1
1/5 Test  #9: PET_TESTS_CPLUSPLUS_1 ............   Passed   56.74 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
2/5 Test #10: PET_TESTS_CPLUSPLUS_4 ............   Passed   63.69 sec
    Start 11: PET_TESTS_PYTHON
3/5 Test #11: PET_TESTS_PYTHON .................   Passed  105.89 sec
    Start 12: PET_PYTHON_ALGEBRA
4/5 Test #12: PET_PYTHON_ALGEBRA ...............   Passed    9.77 sec
    Start 13: PET_PYTHON_OBJFUN
5/5 Test #13: PET_PYTHON_OBJFUN ................   Passed    1.84 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 760.05 sec
KrisThielemans commented 2 years ago

ok. could make a env_sirf.cmd.in then.

I think this indicates we do miss something in our env_sirf.PS1 though. Just not sure what it is. The only way I now to track it down is what I have suggested already.

KrisThielemans commented 2 years ago

Aside from that, if you go the conda way, I would highly recommend using conda for fftw, boost and itk. Ideally HDF5 as well but that might fail.

paskino commented 2 years ago

I've just read here that

Note that on Windows, from Python 3.9 onwards, DLL paths must be added by calling os.add_dll_directory() within Python rather than by setting PATH.

Indeed, in my tests I'm using python 3.10. This would explain all my failed attempts in using environment variables.

More on this

https://docs.python.org/3/library/os.html#os.add_dll_directory https://stackoverflow.com/questions/67805339/is-the-function-os-add-dll-directory-adding-directories-permanently

paskino commented 2 years ago

Some more discussion on Python add_dll_directory here https://discourse.vtk.org/t/how-to-write-a-custom-function-that-uses-vtk-c-and-then-use-it-in-python/9030/7?u=edoardo_pasca

KrisThielemans commented 2 years ago

that cookie stuff referenced there is just ridiculously complicated. sigh. I guess you'd have to make a persistent cookie somehow (no idea how to do that in Python), as we surely don't want to tell people to call add_dll_directory explicitly like that whenever they want to import sirf.