Closed zhang-qiang-github closed 4 years ago
@zhang-qiang-github, please add more details about your extraction so we can better understand your error.
I am sorry for not clear about those information.
The code to use radiomics is:
from radiomics import featureextractor
extractor = featureextractor.RadiomicsFeatureExtractor()
result = extractor.execute(imageFilepath=os.path.join(setting['tempFolder'], 'image.nrrd'), maskFilepath=os.path.join(setting['tempFolder'], 'mask.nrrd'))
I have print the result of radiomics, and it's clear that the radiomics calculated 129 features when I run it in python. However, after pyinstaller, the radiomics only calculated 22 features, which make me very confused.
please run with logging:
in interactive mode:
import logging
import radiomics
radiomics.setVerbosity(logging.INFO)
or on the commandline, add option -v 4
(logging to console) and/or --logging-level INFO --log-file pyradiomics.log
(logging to file)
See also the PyRadiomics documentation
when run the code in python, the log is:
No valid config parameter, using defaults: {'minimumROIDimensions': 2, 'minimumR
OISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None,
'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False,
'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 're
segmentRange': None, 'label': 1, 'additionalInfo': True}
Enabled image types: {'Original': {}}
Enabled features: {'firstorder': [], 'glcm': [], 'gldm': [], 'glrlm': [], 'glszm
': [], 'ngtdm': [], 'shape': []}
Calculating features with label: 1
Loading image and mask
Computing shape
Adding image type "Original" with custom settings: {}
Calculating features for original image
Computing firstorder
Computing glcm
GLCM is symmetrical, therefore Sum Average = 2 * Joint Average, only 1 needs to
be calculated
Computing gldm
Computing glrlm
Computing glszm
Computing ngtdm
When run the exe after pyinstall, the log is:
No valid config parameter, using defaults: {'minimumROIDimensions': 2, 'minimumR
OISize': None, 'normalize': False, 'normalizeScale': 1, 'removeOutliers': None,
'resampledPixelSpacing': None, 'interpolator': 'sitkBSpline', 'preCrop': False,
'padDistance': 5, 'distances': [1], 'force2D': False, 'force2Ddimension': 0, 're
segmentRange': None, 'label': 1, 'additionalInfo': True}
Enabled image types: {'Original': {}}
Enabled features: {}
Calculating features with label: 1
Loading image and mask
Adding image type "Original" with custom settings: {}
Calculating features for original image
Also, I have upload the test code in github. You can test it. Environment is: win7 python 3.7.0 pyradiomics 2.2.0 pyinstaller 3.5
@zhang-qiang-github it appears that there are no features enabled
Enabled features: {}
However, by default, all feature classes should be enabled if you don't use a customization file.
I'm not entirely sure what is happening, but PyRadiomics does determine the available feature classes dynamically, by reading the radiomics folder. It is possible that due to the packaging by pyinstaller, this does not work properly.
Can you run using one of the example parameter files? That way, you explicitly enable feature classes. If for some reason, PyRadiomics is unable to access them, it will throw an error stating that the requested feature class is not found.
On a side note, is the a specific reason for using pyinstaller to run an extraction? PyRadiomics is packaged with a executable command line interface, and per-built wheels are available for Linux, Mac and Windows.
EDIT: If you want to ensure you have a static snapshot of the package to make reproducing your extraction easier, PyRadiomics also auto-builds docker images, including one that exposes the CLI script.
Update:
It may be that PyInstaller does not know that PyRadiomics needs the feature class modules, as these are not imported by regular import statements.
In the pyinstaller docs you can read:
What other modules and libraries does your script need in order to run? (These are sometimes called its “dependencies”.)
To find out, PyInstaller finds all the import statements in your script. It finds the imported modules and looks in them for import statements, and so on recursively, until it has a complete list of modules your script may use.
(...)
Some Python scripts import modules in ways that PyInstaller cannot detect: for example, by using the
__import__()
function with variable data, using imp.find_module(), or manipulating the sys.path value at run time. If your script requires files that PyInstaller does not know about, you must help it:
- You can give additional files on the pyinstaller command line.
- You can give additional import paths on the command line.
- You can edit the myscript.spec file that PyInstaller writes the first time you run it for your script. In the spec file you can tell PyInstaller about code modules that are unique to your script.
- You can write “hook” files that inform PyInstaller of hidden imports. If you create a “hook” for a package that other users might also use, you can contribute your hook file to PyInstaller.
Whereas the feature classes are imported here:
def getFeatureClasses():
"""
Iterates over all modules of the radiomics package using pkgutil and subsequently imports those modules.
Return a dictionary of all modules containing featureClasses, with modulename as key, abstract
class object of the featureClass as value. Assumes only one featureClass per module
This is achieved by inspect.getmembers. Modules are added if it contains a member that is a class,
with name starting with 'Radiomics' and is inherited from :py:class:`radiomics.base.RadiomicsFeaturesBase`.
This iteration only runs once (at initialization of toolbox), subsequent calls return the dictionary created by the
first call.
"""
global _featureClasses
if _featureClasses is None: # On first call, enumerate possible feature classes and import PyRadiomics modules
_featureClasses = {}
for _, mod, _ in pkgutil.iter_modules([os.path.dirname(__file__)]):
if str(mod).startswith('_'): # Skip loading of 'private' classes, these don't contain feature classes
continue
__import__('radiomics.' + mod)
module = sys.modules['radiomics.' + mod]
attributes = inspect.getmembers(module, inspect.isclass)
for a in attributes:
if a[0].startswith('Radiomics'):
for parentClass in inspect.getmro(a[1])[1:]: # only include classes that inherit from RadiomicsFeaturesBase
if parentClass.__name__ == 'RadiomicsFeaturesBase':
_featureClasses[mod] = a[1]
break
return _featureClasses
We recommend some doctor to use the radiomics, but they can not install pyradiomics. Thus, we use pyinstaller to generate exe, and doctor can directly get the results.
I'am not sure how to run the code by using the example parameter files. Do you mean:
from radiomics import featureextractor
extractor = featureextractor.RadiomicsFeatureExtractor("Params.yaml")
result = extractor.execute(imageFilepath='tempFolder\image.nrrd', maskFilepath='tempFolder\mask.nrrd')
keys = list(result.keys())
for index, key in enumerate(keys):
print(index, key)
When I put the "Params.yaml" in the folder with exe, bug happen:
C:\Users\zhq\Desktop\test\dist\test>test.exe
Traceback (most recent call last):
File "test.py", line 2, in <module>
File "site-packages\radiomics\featureextractor.py", line 60, in __init__
File "site-packages\radiomics\featureextractor.py", line 166, in _applyParams
File "site-packages\pykwalify\core.py", line 84, in __init__
pykwalify.errors.CoreError: <CoreError: error code 3: Provided source_file do no
t exists on disk : C:\Users\zhq\Desktop\test\dist\test\radiomics\schemas\paramSc
hema.yaml: Path: '/'>
[8500] Failed to execute script test
Then, I find the folder "schemas" and put it in the correct path. However, bug still exists:
Traceback (most recent call last):
File "test.py", line 2, in <module>
File "site-packages\radiomics\featureextractor.py", line 60, in __init__
File "site-packages\radiomics\featureextractor.py", line 167, in _applyParams
File "site-packages\pykwalify\core.py", line 155, in validate
File "site-packages\pykwalify\core.py", line 202, in _start_validate
File "site-packages\pykwalify\core.py", line 236, in _validate
File "site-packages\pykwalify\core.py", line 570, in _validate_mapping
File "site-packages\pykwalify\core.py", line 236, in _validate
File "site-packages\pykwalify\core.py", line 496, in _validate_mapping
File "site-packages\pykwalify\core.py", line 259, in _handle_func
File "C:\Users\zhq\Desktop\test\dist\test\radiomics\schemas\schemaFuncs.py", l
ine 61, in checkFeatureClass
'Feature Class %s is not recognized. Available feature classes are %s' % (cl
assName, list(featureClasses.keys())))
ValueError: Feature Class shape is not recognized. Available feature classes are
[]
[26152] Failed to execute script test
On the other hand, the bug may be caused by lacking some files of radiomics. Do you know which files are lacked? I think I can manually copy those files.
@JoostJM The stranger thing is: when I run the code in python, the _featureClasses returned from getFeatureClasses() contains 8 modules including 'firstorder', 'glcm', 'gldm', 'glrlm', 'glszm', ngtdm', 'shape', 'shape2D'. However, after pyinstaller, the _featureClasses returned from getFeatureClasses() contains 0 modules. I don't know the reason.
As you can read from my update comment, your issues are caused by the fact that pyinstaller is unable to detect all the files PyRadiomics needs.
If you want to recommend some doctor to use pyradiomics, but don't know how to use Python/command line, I think it is easier to use PyRadiomics via the Slicer extension SlicerRadiomics.
@JoostJM Do you know what files of pyradiomics should be included by pyinstaller? I can manually copy those files.
Dear @JoostJM , I think I may find the reason for this problem. In the getFeatureClasses(), the program would find all modules from the path os.path.dirname(file). In path, the os.path.dirname(file) would be: "D:\Python\Python37\Lib\site-packages\radiomics". However, when run the exe generated by pyinstaller, the os.path.dirname(file) is "C:\Users\zhq\Desktop\test\dist\test\radiomics". Thus, the program cannot find the necessary modules. In this way, I specify the os.path.dirname(file) to be "D:\Python\Python37\Lib\site-packages\radiomics". bug appear: "ModuleNotFoundError: No module named "radiomics.base"
@zhang-qiang-github, basically, you'd need everything in the radiomics
folder and it's subfolders.
As to changing the dirname to be the other, I'd strongly advise against it, as it is mixing up 2 different Python installations, with all the possible errors/bugs. If it works, it's the exception.
If you really must use pyinstaller, I'd advise to dive deep into the documentation of pyinstaller and see how to configure it so it works for PyRadiomics. Still, I'd advise to use the Slicer extension if you want clinicians/non-programmers to work with PyRadiomics. Even if you package PyRadiomics in pyinstaller, it still requires your user to understand how to work with the command line.
@zhang-qiang-github Have you been able to solve your issue?
@JoostJM I find a very stupid solution. In the init of pyradiomics, I directly import all the modules instead of dynamically importing. Those, the exe after pyinstaller would be OK.
@zhang-qiang-github that works, but is not the spirit of PyRadiomics. The dynamic import is there to simplify the addition of new or experimental modules (i.e. you just have to dump the module file in the folder). Directly importing works, but requires you to update it everytime you make changes to the featureclasses.
@zhang-qiang-github, beware however, that this may fix your issue with the feature classes, but does not fix the problem of additional missing files (in this case, mainly the files needed for parameter file validation). I still recommend you dive into the docs of PyInstaller to see how to add the additional files (basically, everything inside the radiomics
folder, with the exception of the C source code (radiomics/src
))
Dear @JoostJM, I have encountered the same question as @zhang-qiang-github, is there any solution now?
Hi, this seems to be a relatively old thread but I also got the same problem and solved by using the parameters file and incorporating the radiomics folder into the pyinstaller as suggested by Joost, see example below:
added_files = [ ( './pyradiomics-master', '.' ), ]
a = Analysis( ['helloRadiomics_ML.py'], pathex=[], binaries=[], datas=added_files, hiddenimports=[], hookspath=[], hooksconfig={}, runtime_hooks=[], excludes=[], win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher, noarchive=False, )
@zhang-qiang-github I had try "--collect-submodules". For example, my codes need to import "pydicom" and "radiomics".
pyinstaller -F --collect-submodules=pydicom --collect-submodules=radiomics codes.py
hope that can help.
When I run the code in pycharm, the results are OK. However, after pyinstaller, the pyradiomics can only output 21 results rather than 129.