search

AIM-Harvard / pyradiomics

Open-source python package for the extraction of Radiomics features from 2D and 3D images and binary masks. Support: https://discourse.slicer.org/c/community/radiomics
http://pyradiomics.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1.16k stars 500 forks source link

CLI: Segfault 11 under MacOS #452

Closed dfsp-spirit closed 5 years ago

dfsp-spirit commented 5 years ago

I'm new to pyradiomics and just installed it under MacOS/conda:

$ conda install -y pip
$ pip install pyradiomics
$ python -c 'import radiomics; print(radiomics.__version__)'
2.1.2
$ python --version
Python 3.7.0

So far, everthing looks fine.

But the command line interface segfaults (I use the repo checkout to get the demo data only):

$ git clone https://github.com/Radiomics/pyradiomics pyradiomics_repo
$ pyradiomics pyradiomics_repo/data/brain1_image.nrrd pyradiomics_repo/data/brain1_label.nrrd
Segmentation fault: 11
$

Am I doing something wrong with the command line? Is there any logfile I should check?

JoostJM commented 5 years ago

Hi, if you run

pyradiomics pyradiomics_repo/data/brain1_image.nrrd pyradiomics_repo/data/brain1_label.nrrd --logging-level DEBUG --log-file pyradiomics.log a log file is created. If you can share that file, it'll be easier to track down the error.

dfsp-spirit commented 5 years ago

Hm, the error is:

ValueError: Error reading image Filepath or SimpleITK object

Strange, the error seems to suggest that it cannot read the image. Here is the full log file:

[2019-01-09 14:04:05] D: radiomics.script: Logging initialized [2019-01-09 14:04:05] I: radiomics.script: Starting PyRadiomics (version: 2.1.2) [2019-01-09 14:04:05] I: radiomics.script: Processing input... [2019-01-09 14:04:05] I: radiomics.script: Input valid, starting sequential extraction from 1 case(s)... [2019-01-09 14:04:05] D: radiomics.script: No overrides found [2019-01-09 14:04:05] I: radiomics.script: Processing case 1 [2019-01-09 14:04:05] I: radiomics.featureextractor: No valid config parameter, applying defaults: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:04:05] I: radiomics.featureextractor: Enabled image types: {'Original': {}} [2019-01-09 14:04:05] I: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:04:05] I: radiomics.featureextractor: Calculating features with label: 1 [2019-01-09 14:04:05] D: radiomics.featureextractor: Enabled images types: {'Original': {}} [2019-01-09 14:04:05] D: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:04:05] D: radiomics.featureextractor: Current settings: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:04:05] I: radiomics.featureextractor: Loading image and mask [2019-01-09 14:04:05] E: radiomics.script: Feature extraction failed! Traceback (most recent call last): File "/Users/user/develop/brainload/env/lib/python3.7/site-packages/radiomics/scripts/segment.py", line 40, in extractSegment feature_vector.update(extractor.execute(imageFilepath, maskFilepath, label)) File "/Users/user/develop/brainload/env/lib/python3.7/site-packages/radiomics/featureextractor.py", line 393, in execute image, mask = self.loadImage(imageFilepath, maskFilepath) File "/Users/user/develop/brainload/env/lib/python3.7/site-packages/radiomics/featureextractor.py", line 496, in loadImage raise ValueError('Error reading image Filepath or SimpleITK object') ValueError: Error reading image Filepath or SimpleITK object [2019-01-09 14:04:05] I: radiomics.script: Processing results... [2019-01-09 14:04:05] I: radiomics.script: Finished extraction successfully... [2019-01-09 14:04:15] D: radiomics.script: Logging initialized [2019-01-09 14:04:15] I: radiomics.script: Starting PyRadiomics (version: 2.1.2) [2019-01-09 14:04:15] I: radiomics.script: Processing input... [2019-01-09 14:04:15] I: radiomics.script: Input valid, starting sequential extraction from 1 case(s)... [2019-01-09 14:04:15] D: radiomics.script: No overrides found [2019-01-09 14:04:15] I: radiomics.script: Processing case 1 [2019-01-09 14:04:15] I: radiomics.featureextractor: No valid config parameter, applying defaults: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:04:15] I: radiomics.featureextractor: Enabled image types: {'Original': {}} [2019-01-09 14:04:15] I: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:04:15] I: radiomics.featureextractor: Calculating features with label: 1 [2019-01-09 14:04:15] D: radiomics.featureextractor: Enabled images types: {'Original': {}} [2019-01-09 14:04:15] D: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:04:15] D: radiomics.featureextractor: Current settings: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:04:15] I: radiomics.featureextractor: Loading image and mask [2019-01-09 14:04:15] D: radiomics.imageoperations: Checking mask with label 1 [2019-01-09 14:04:15] D: radiomics.imageoperations: Calculating bounding box [2019-01-09 14:04:15] D: radiomics.imageoperations: Checking minimum number of dimensions requirements (2) [2019-01-09 14:04:15] D: radiomics.featureextractor: Image and Mask loaded and valid, starting extraction [2019-01-09 14:04:15] D: radiomics.imageoperations: Cropping to size [47 70 7] [2019-01-09 14:05:36] D: radiomics.script: Logging initialized [2019-01-09 14:05:36] I: radiomics.script: Starting PyRadiomics (version: 2.1.2) [2019-01-09 14:05:36] I: radiomics.script: Processing input... [2019-01-09 14:05:36] I: radiomics.script: Input valid, starting sequential extraction from 1 case(s)... [2019-01-09 14:05:36] D: radiomics.script: No overrides found [2019-01-09 14:05:36] I: radiomics.script: Processing case 1 [2019-01-09 14:05:36] I: radiomics.featureextractor: No valid config parameter, applying defaults: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:05:36] I: radiomics.featureextractor: Enabled image types: {'Original': {}} [2019-01-09 14:05:36] I: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:05:36] I: radiomics.featureextractor: Calculating features with label: 1 [2019-01-09 14:05:36] D: radiomics.featureextractor: Enabled images types: {'Original': {}} [2019-01-09 14:05:36] D: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:05:36] D: radiomics.featureextractor: Current settings: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:05:36] I: radiomics.featureextractor: Loading image and mask [2019-01-09 14:05:36] D: radiomics.imageoperations: Checking mask with label 1 [2019-01-09 14:05:36] D: radiomics.imageoperations: Calculating bounding box [2019-01-09 14:05:36] D: radiomics.imageoperations: Checking minimum number of dimensions requirements (2) [2019-01-09 14:05:36] D: radiomics.featureextractor: Image and Mask loaded and valid, starting extraction [2019-01-09 14:05:36] D: radiomics.imageoperations: Cropping to size [47 70 7] [2019-01-09 14:05:45] D: radiomics.script: Logging initialized [2019-01-09 14:05:45] I: radiomics.script: Starting PyRadiomics (version: 2.1.2) [2019-01-09 14:05:45] I: radiomics.script: Processing input... [2019-01-09 14:05:45] I: radiomics.script: Input valid, starting sequential extraction from 1 case(s)... [2019-01-09 14:05:45] D: radiomics.script: No overrides found [2019-01-09 14:05:45] I: radiomics.script: Processing case 1 [2019-01-09 14:05:45] I: radiomics.featureextractor: No valid config parameter, applying defaults: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:05:45] I: radiomics.featureextractor: Enabled image types: {'Original': {}} [2019-01-09 14:05:45] I: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:05:45] I: radiomics.featureextractor: Calculating features with label: 1 [2019-01-09 14:05:45] D: radiomics.featureextractor: Enabled images types: {'Original': {}} [2019-01-09 14:05:45] D: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:05:45] D: radiomics.featureextractor: Current settings: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:05:45] I: radiomics.featureextractor: Loading image and mask [2019-01-09 14:05:45] D: radiomics.imageoperations: Checking mask with label 1 [2019-01-09 14:05:45] D: radiomics.imageoperations: Calculating bounding box [2019-01-09 14:05:45] D: radiomics.imageoperations: Checking minimum number of dimensions requirements (2) [2019-01-09 14:05:45] D: radiomics.featureextractor: Image and Mask loaded and valid, starting extraction [2019-01-09 14:05:45] D: radiomics.imageoperations: Cropping to size [47 70 7]

JoostJM commented 5 years ago

@dfsp-spirit That was only the first run, the others it crashed after cropping. I think it is a problem in the C extensions. Can you build a python 3.6 environment and try again? currently PyRadiomics is only pre-compiled for python 2.7, 3.4, 3.5 and 3.6

dfsp-spirit commented 5 years ago

Hm, does not make a difference. Here is what I did, copied from my bash history:

620  conda create -y --name pyradiomics python=3.6
621  conda activate pyradiomics
622  pip install pyradiomics
623  pyradiomics brain1_image.nrrd brain1_label.nrrd --logging-level DEBUG --log-file pyradiomics.log

And it still segfaults. Could it be the data I am using?

Or another idea: is the CLI tested during CI tests? Maybe I could try a command line from the tests to see whether it has to do with my system or dependencies. At a quick glance I only found unit tests.

I started a fresh log before running the command again, here is the full log file:

[2019-01-09 14:24:15] D: radiomics.script: Logging initialized [2019-01-09 14:24:15] I: radiomics.script: Starting PyRadiomics (version: 2.1.2) [2019-01-09 14:24:15] I: radiomics.script: Processing input... [2019-01-09 14:24:15] I: radiomics.script: Input valid, starting sequential extraction from 1 case(s)... [2019-01-09 14:24:15] D: radiomics.script: No overrides found [2019-01-09 14:24:15] I: radiomics.script: Processing case 1 [2019-01-09 14:24:15] I: radiomics.featureextractor: No valid config parameter, applying defaults: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:24:15] I: radiomics.featureextractor: Enabled image types: {'Original': {}} [2019-01-09 14:24:15] I: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:24:15] I: radiomics.featureextractor: Calculating features with label: 1 [2019-01-09 14:24:15] D: radiomics.featureextractor: Enabled images types: {'Original': {}} [2019-01-09 14:24:15] D: radiomics.featureextractor: Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm': [], 'ngtdm': [], 'shape': []} [2019-01-09 14:24:15] D: radiomics.featureextractor: Current settings: {'minimumROIDimensions': 2, 'minimumROISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None, 'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False, 'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 'resegmentRange': None, 'label': 1, 'additionalInfo': True} [2019-01-09 14:24:15] I: radiomics.featureextractor: Loading image and mask [2019-01-09 14:24:15] D: radiomics.imageoperations: Checking mask with label 1 [2019-01-09 14:24:15] D: radiomics.imageoperations: Calculating bounding box [2019-01-09 14:24:15] D: radiomics.imageoperations: Checking minimum number of dimensions requirements (2) [2019-01-09 14:24:15] D: radiomics.featureextractor: Image and Mask loaded and valid, starting extraction [2019-01-09 14:24:15] D: radiomics.imageoperations: Cropping to size [47 70 7]

The file load error is gone.

JoostJM commented 5 years ago

@dfsp-spirit currently command line is not tested. But you could try to run the helloRadiomics.py example contained in the repo. Does that work?

dfsp-spirit commented 5 years ago

No:

(pyradiomics) [user@BloodyiMac4:~/pyradiomics_repo/examples] $ python helloRadiomics.py
Calculating features
Segmentation fault: 11
(pyradiomics) [user@BloodyiMac4:~/pyradiomics_repo/examples] $

I guess it has to do with my system then. Thanks for your quick replies and effort. I will try under Linux instead of MacOS, so I can get rid of conda. It adds another layer that could be the cause of all this trouble. I will report back when I get it to work.

JoostJM commented 5 years ago

@fedorov, can you try to reproduce the error on Mac?

JoostJM commented 5 years ago

@dfsp-spirit, I guess something is going wrong with the C extension. Sadly I'm unable to reproduce the error, as I'm developping on Windows and Linux. Still, the unit test should suffer the same problem on Mac (Travis CI unit tests), but I did not see an error there either... Even the conda distribution is working fine there (I'm currently having some issues on Linux and Windows conda)

P.S. can you try a pypi (ie pip) install of PyRadiomics in a python 3.6 environment (should also be possible in conda)?

dfsp-spirit commented 5 years ago

It was MacOS High Sierra 10.13.6 btw. With conda3, as the system python that comes with that OS is more or less useless.

dfsp-spirit commented 5 years ago

I got it to work under Linux. Most likely the error is related to conda. This was under Ubuntu 16.04 LTS:

python3 -m venv venv_pyradiomics
source venv_pyradiomics/bin/activate
pip3 install pyradiomics
git clone https://github.com/Radiomics/pyradiomics pyradiomics_repo
cd pyradiomics_repo/examples/
python3 helloRadiomics.py

...and it works. The output is:

$ python3 helloRadiomics.py Calculating features Computed diagnostics_Versions_PyRadiomics: 2.1.2 Computed diagnostics_Versions_Numpy: 1.15.4 Computed diagnostics_Versions_SimpleITK: 1.2.0 Computed diagnostics_Versions_PyWavelet: 1.0.0 Computed diagnostics_Versions_Python: 3.5.2 Computed diagnostics_Configuration_Settings: {'interpolator': 3, 'additionalInfo': True, 'normalizeScale': 1, 'removeOutliers': None, 'resegmentRange': None, 'distances': [1], 'minimumROIDimensions': 2, 'force2Ddimension': 0, 'minimumROISize': None, 'force2D': False, 'normalize': False, 'label': 1, 'preCrop': False, 'binWidth': 25, 'resampledPixelSpacing': None, 'padDistance': 5} Computed diagnostics_Configuration_EnabledImageTypes: {'Original': {}} Computed diagnostics_Image-original_Hash: 5c9ce3ca174f0f8324aa4d277e0fef82dc5ac566 Computed diagnostics_Image-original_Spacing: (0.7812499999999999, 0.7812499999999999, 6.499999999999998) Computed diagnostics_Image-original_Size: (256, 256, 25) Computed diagnostics_Image-original_Mean: 385.6564080810547 Computed diagnostics_Image-original_Minimum: 0.0 Computed diagnostics_Image-original_Maximum: 3057.0 Computed diagnostics_Mask-original_Hash: 9dc2c3137b31fd872997d92c9a92d5178126d9d3 Computed diagnostics_Mask-original_Spacing: (0.7812499999999999, 0.7812499999999999, 6.499999999999998) Computed diagnostics_Mask-original_Size: (256, 256, 25) Computed diagnostics_Mask-original_BoundingBox: (162, 84, 11, 47, 70, 7) Computed diagnostics_Mask-original_VoxelNum: 4137 Computed diagnostics_Mask-original_VolumeNum: 2 Computed diagnostics_Mask-original_CenterOfMassIndex: (186.98549673676578, 106.3562968334542, 14.38917089678511) Computed diagnostics_Mask-original_CenterOfMass: (46.47304432559825, 16.518518098863908, 15.529610829103234) Computed original_firstorder_Mean: 825.2354363065023 Computed original_firstorder_Skewness: 0.27565085908587594 (venv_pyradiomics) [u@h:~/pyradiomics_repo/examples] $

dfsp-spirit commented 5 years ago

P.S. can you try a pypi (ie pip) install of PyRadiomics in a python 3.6 environment (should also be possible in conda)?

Is that not what I did in my 3rd post? If not, please let me know what I should do. I will gladly give it a try.

JoostJM commented 5 years ago

Sorry my bad. In that case you could try the conda package in the py 3.6 environment. See here conda install -c radiomics pyradiomics

dfsp-spirit commented 5 years ago 
 510  conda create -y --name pyradiomics_conda python=3.6
 511  conda activate pyradiomics_conda
 512  conda install -c radiomics pyradiomics

The last command gives me:

$ conda install -c radiomics pyradiomics Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

Current channels:

To search for alternate channels that may provide the conda package you're looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

(pyradiomics_conda) [user@BloodyiMac4:~] $

I can see that a suitable pyradiomics exists at https://anaconda.org/Radiomics/pyradiomics/files though. Dunno about simpleitk. Both

conda install simpleitk

and 

conda install -c Radiomics simpleitk

fail with the same error. Maybe that is the problem.

Is the following line you have in your conda recipe:

 - SimpleITK !=1.1.0

intended as it is?

dfsp-spirit commented 5 years ago

Oh, wait: you have to:

conda install -c simpleitk simpleitk

Then:

conda install -c Radiomics pyradiomics

And then, the conda package works and does not segfault anymore. Great! Problem solved. I will use conda to run pyradiomics under MacOS.

I'm not sure whether you can update the recipe so users do not have to do this? From my perspective, this can be closed. Thanks a lot for your quick help. Really appreciate it.

EDIT: It may be worth nothing that some of the other dependencies require the conda-forge channel. So unless you already have it, you also need to do:

conda config --add channels conda-forge

before trying to install pyradiomics.