Open marcmorcos opened 2 years ago
Hello @marcmorcos,
sounds an incorrect Python environment was loaded, in which MIScnn is not installed.
Please verify if your pip is connected to the Python 3.9.1 environment you are using. In older linux OS environments pip is connected to Python 2.7 instead of 3.X, which can lead to installation mistakes.
Another way to install Python packages with pip is by calling pip directly from your Python environment like
python -m pip install miscnn
Cheers, Dominik
Thank you @muellerdo. Unfortunately, I think the problem lies elsewhere. I've tried to install on 3 different computers (1 linux cluster, 1 linux laptop, and 1 windows 10 PC).
I've had a few different types of issues installing as well. They seem related to dependency mismatches with required packages (i.e. numpy 1.19.2 not compatible with pillow and/or tensorflow if I recall correctly; similar issue with pydicom).
Is this something you have encountered?
Is this something you have encountered?
Mhm, have not encountered it before, sadly.
Could you try it in a fresh Python 3.7 or Python 3.8 environment? Currently, we only test MIScnn with Python 3.7 and Python 3.8 in our automatic build/unittesting infrastructure.
ve had a few different types of issues installing as well. They seem related to dependency mismatches with required packages (i.e. numpy 1.19.2 not compatible with pillow and/or tensorflow if I recall correctly; similar issue with pydicom).
Could you please attach the (error) log of the installation here?
Sure here's the output after a fresh environment on Windows 10. (Python=3.7.11, pip=21.0.1)
conda create -n miscnn python=3.7
conda activate miscnn
python -m pip install miscnn
No install errors w/ Python3.7 on Win10. Will report back on Linux install errors when on my other PC.
python -c "from miscnn.data_loading.interfaces.dicom_io import DICOM_interface"
2021-09-28 07:37:25.344946: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2021-09-28 07:37:25.345126: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\__init__.py", line 21, in <module>
from miscnn.processing.preprocessor import Preprocessor
File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\processing\preprocessor.py", line 32, in <module>
from miscnn.processing.data_augmentation import Data_Augmentation
File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\processing\data_augmentation.py", line 27, in <module>
from batchgenerators.dataloading import SingleThreadedAugmenter
ImportError: cannot import name 'SingleThreadedAugmenter' from 'batchgenerators.dataloading' (C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\batchgenerators\dataloading\__init__.py)
Same error when calling "import miscnn".
Thanks for your help. Will report back from the Linux install.
From linux computing cluster. [Python=3.7.9; pip 20.0.2 from /home/mmorcos/miscnn/lib/python3.7/site-packages/pip (python 3.7)]
module load python/3.7.9
virtualenv miscnn
source miscnn/bin/activate
python -m pip install miscnn
We get the following:
...
Failed to build pillow
ERROR: pandas 1.3.0+computecanada has requirement numpy>=1.20.0, but you'll have numpy 1.19.2+computecanada which is incompatible.
...
Installing collected packages: tqdm, pillow, numpy, six, cycler, pyparsing, kiwisolver, python-dateutil, certifi, matplotlib, SimpleITK, scipy, PyWavelets, imageio, tifffile, networkx, scikit-image, joblib, threadpoolctl, scikit-learn, argparse, linecache2, traceback2, unittest2, future, batchgenerators, cached-property, h5py, keras-preprocessing, keras-nightly, google-pasta, protobuf, tensorflow-estimator, absl-py, grpcio, wheel, termcolor, werkzeug, tensorboard-plugin-wit, typing-extensions, zipp, importlib-metadata, markdown, tensorboard-data-server, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, urllib3, charset-normalizer, idna, requests, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, astunparse, opt-einsum, wrapt, flatbuffers, gast, tensorflow, pydicom, pytz, pandas, packaging, nibabel, typeguard, tensorflow-addons, miscnn
Running setup.py install for pillow ... error
ERROR: Command errored out with exit status 1:
command: /home/mmorcos/miscnn/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mygrsrfs/pillow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mygrsrfs/pillow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yure9ba6/install-record.txt --single-version-externally-managed --compile --install-headers /home/mmorcos/miscnn/include/site/python3.7/pillow
cwd: /tmp/pip-install-mygrsrfs/pillow/
Complete output (175 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/PIL
copying src/PIL/features.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/_version.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/_util.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/_tkinter_finder.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/_binary.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/__main__.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/__init__.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/XpmImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/XbmImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/XVThumbImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/WmfImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/WebPImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/WalImageFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/TiffTags.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/TiffImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/TgaImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/TarIO.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/SunImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/SpiderImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/SgiImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PyAccess.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PsdImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PpmImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PngImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PixarImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PdfParser.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PdfImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PcxImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PcfFontFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PcdImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PalmImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PaletteFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/PSDraw.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/MspImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/MpoImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/MpegImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/MicImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/McIdasImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/JpegPresets.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/JpegImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/Jpeg2KImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/IptcImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImtImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageWin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageTransform.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageTk.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageStat.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageShow.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageSequence.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageQt.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImagePath.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImagePalette.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageOps.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageMorph.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageMode.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageMath.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageGrab.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageFont.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageFilter.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageEnhance.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageDraw2.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageDraw.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageColor.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageCms.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImageChops.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/Image.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ImImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/IcoImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/IcnsImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/Hdf5StubImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GribStubImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GimpPaletteFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GimpGradientFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GifImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GdImageFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/GbrImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/FtexImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/FpxImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/FontFile.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/FliImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/FitsStubImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ExifTags.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/EpsImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/DdsImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/DcxImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/CurImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/ContainerIO.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/BufrStubImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/BmpImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/BlpImagePlugin.py -> build/lib.linux-x86_64-3.7/PIL
copying src/PIL/BdfFontFile.py -> build/lib.linux-x86_64-3.7/PIL
running egg_info
writing src/Pillow.egg-info/PKG-INFO
writing dependency_links to src/Pillow.egg-info/dependency_links.txt
writing top-level names to src/Pillow.egg-info/top_level.txt
reading manifest file 'src/Pillow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.c'
warning: no files found matching '*.h'
warning: no files found matching '*.sh'
warning: no previously-included files found matching '.appveyor.yml'
warning: no previously-included files found matching '.clang-format'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.editorconfig'
warning: no previously-included files found matching '.readthedocs.yml'
warning: no previously-included files found matching 'codecov.yml'
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
no previously-included directories found matching '.ci'
writing manifest file 'src/Pillow.egg-info/SOURCES.txt'
running build_ext
The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
Traceback (most recent call last):
File "/tmp/pip-install-mygrsrfs/pillow/setup.py", line 1020, in <module>
zip_safe=not (debug_build() or PLATFORM_MINGW),
File "/home/mmorcos/miscnn/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/mmorcos/miscnn/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/mmorcos/miscnn/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
_build_ext.run(self)
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.7.9/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/tmp/pip-install-mygrsrfs/pillow/setup.py", line 788, in build_extensions
raise RequiredDependencyException(f)
__main__.RequiredDependencyException: jpeg
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-mygrsrfs/pillow/setup.py", line 1033, in <module>
raise RequiredDependencyException(msg)
__main__.RequiredDependencyException:
The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
----------------------------------------
ERROR: Command errored out with exit status 1: /home/mmorcos/miscnn/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mygrsrfs/pillow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mygrsrfs/pillow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yure9ba6/install-record.txt --single-version-externally-managed --compile --install-headers /home/mmorcos/miscnn/include/site/python3.7/pillow Check the logs for full command output.
The first error seems like a dependency version mismatch. The second error may be due to a missing libjpeg on the cluster. I get similar error (first error) on my personal linux laptop.
Sure here's the output after a fresh environment on Windows 10. (Python=3.7.11, pip=21.0.1)
conda create -n miscnn python=3.7 conda activate miscnn python -m pip install miscnn
No install errors w/ Python3.7 on Win10. Will report back on Linux install errors when on my other PC.
python -c "from miscnn.data_loading.interfaces.dicom_io import DICOM_interface" 2021-09-28 07:37:25.344946: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found 2021-09-28 07:37:25.345126: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\__init__.py", line 21, in <module> from miscnn.processing.preprocessor import Preprocessor File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\processing\preprocessor.py", line 32, in <module> from miscnn.processing.data_augmentation import Data_Augmentation File "C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\miscnn\processing\data_augmentation.py", line 27, in <module> from batchgenerators.dataloading import SingleThreadedAugmenter ImportError: cannot import name 'SingleThreadedAugmenter' from 'batchgenerators.dataloading' (C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\batchgenerators\dataloading\__init__.py)
Same error when calling "import miscnn".
Thanks for your help. Will report back from the Linux install.
For Windows, this issue seems to be resolved by downgrading batchgenerators to 0.21 after pip installing miscnn
Thanks for the error logs!
Could you please try downloading this git repository (master branch) and try out the following
cd miscnn
python -m pip install --upgrade pip
python -m pip install pytest
pip install -r requirements.txt
pytest -v
Also I successfully tested the installation & pytest of the current master branch with Python 3.8 in a Ubuntu:18.04 docker container without errors anymore. Including our CI / automatic GitHub Action building system for Python 3.7 & Python 3.8.
ImportError: cannot import name 'SingleThreadedAugmenter' from 'batchgenerators.dataloading' (C:\Users\Marc Morcos\miniconda3\envs\miscnn\lib\site-packages\batchgenerators\dataloading__init__.py)
This resulted due to the newest batchgenerators version (0.23) changed some functionality. Temporarily, I have now fixed the batchgenerators version to the latest functional version (0.21).
Updated package dependency list:
The second error may be due to a missing libjpeg on the cluster. I get a similar error (first error) on my personal linux laptop.
Mhm. Since we are working with images and pillow is one of the few foundation packages for image processing in Python, it is probably necessary to install Zlib & libjpeg on your cluster.
Could you please try to install the newest version 1.2.1 of miscnn via pip and to have a look if it's working, now?
Cheers, Dominik
Also having same problem. I'm not seeing version 1.2.1 on pip, though.
ERROR: No matching distribution found for miscnn==1.2.1
Hey @adleberg,
my mistake. Didn't publish the 1.2.1v on PyPI. Now, the 1.2.1v should be downloadable via pip.
Cheers, Dominik
Hello,
I'm following the example for the LCTSC_DICOM. I have installed miscnn using pip (python 3.9.1, pydicom 2.1.2).
The following gives me an error: from miscnn.data_loading.interfaces.dicom_io import DICOM_interface
Have you encountered this before?