search

akhanf / mdt-singularity

Singularity recipe for Microstructure Diffusion Toolbox (http://mdt-toolbox.readthedocs.io/en/latest/index.html)
0 stars 0 forks source link

Problems using the mdt-singularity container #1

Open giorgiag-zz opened 6 years ago

giorgiag-zz commented 6 years ago

Hello @akhanf ,

I am trying to use the singularity container you kindly provided to run some MDT modeling. I installed it and followed the steps but I am getting some errors while running the basic tutorial with the example data provided. I can access the commands, lunch the GUI but at the time of modeling, it crashes. Below is the error i get:

mdt-model-fit 
"BallStick_r1 (Cascade)" b1k_b2k_example_slices_24_38.nii.gz b1k_b2k.prtcl *mask.nii.gz
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
[2018-08-01 12:17:14,001] [INFO] [mdt.utils] [configure_per_model_logging] - Started appending to the per model log file
[2018-08-01 12:17:14,002] [INFO] [mdt.model_fitting] [fit_composite_model] - Using MDT version 0.13.5
[2018-08-01 12:17:14,002] [INFO] [mdt.model_fitting] [fit_composite_model] - Preparing for model S0
[2018-08-01 12:17:14,002] [INFO] [mdt.model_fitting] [fit_composite_model] - Current cascade: ['BallStick_r1 (Cascade)', 'S0']
[2018-08-01 12:17:14,053] [INFO] [mdt.models.composite] [_prepare_input_data] - For this model, S0, we will use a subset of the volumes.
[2018-08-01 12:17:14,054] [INFO] [mdt.models.composite] [_prepare_input_data] - Using 13 out of 103 volumes, indices: [0   1   2   3   4   5  36  37  50  63  76  89 102]
[2018-08-01 12:17:14,055] [INFO] [mdt.utils] [estimate_noise_std] - Trying to estimate a noise std.
[2018-08-01 12:17:14,057] [INFO] [mdt.utils] [estimate_noise_std] - Estimated global noise std 19.613178253173828.
[2018-08-01 12:17:14,058] [INFO] [mdt.model_fitting] [_model_fit_logging] - Fitting S0 model
[2018-08-01 12:17:14,058] [INFO] [mdt.model_fitting] [_model_fit_logging] - The parameters we will fit are: ['S0.s0']
[2018-08-01 12:17:14,058] [INFO] [mdt.model_fitting] [fit_composite_model] - Saving temporary results in output/b1k_b2k_example_slices_24_38_mask/S0/tmp_results.
[2018-08-01 12:17:14,071] [INFO] [mdt.processing_strategies] [_process_chunk] - Computations are at 0.00%, processing next 8865 voxels (8865 voxels in total, 0 processed). Time spent: 0:00:00:00, time left: ? (d:h:m:s).
[2018-08-01 12:17:14,189] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Entered optimization routine.
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Using MOT version 0.5.4
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - We will use a single precision float type for the calculations.
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Using device 'CPU - Intel(R) Xeon(R) CPU           E5607  @ 2.27GHz (Intel(R) OpenCL)'.
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Using compile flags: ['-cl-single-precision-constant', '-cl-denorms-are-zero', '-cl-mad-enable', '-cl-no-signed-zeros']
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - We will use the optimizer Powell with optimizer settings {'reset_method': 'EXTRAPOLATED_POINT', 'patience': 2, 'patience_line_search': 2}
[2018-08-01 12:17:14,190] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Starting optimization preliminaries
[2018-08-01 12:17:14,191] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Finished optimization preliminaries
[2018-08-01 12:17:14,191] [INFO] [mot.cl_routines.optimizing.base] [minimize] - Starting optimization
Traceback (most recent call last):
  File "/usr/local/bin/mdt-model-fit", line 11, in <module>
    sys.exit(ModelFit.console_script())
  File "/usr/local/lib/python3.4/dist-packages/mdt/shell_utils.py", line 44, in console_script
    cls().start(sys.argv[1:])
  File "/usr/local/lib/python3.4/dist-packages/mdt/shell_utils.py", line 63, in start
    self.run(args, {})
  File "/usr/local/lib/python3.4/dist-packages/mdt/cli_scripts/mdt_model_fit.py", line 143, in run
    fit_model()
  File "/usr/local/lib/python3.4/dist-packages/mdt/cli_scripts/mdt_model_fit.py", line 137, in fit_model
    tmp_results_dir=tmp_results_dir)
  File "/usr/local/lib/python3.4/dist-packages/mdt/__init__.py", line 114, in fit_model
    return model_fit.run()
  File "/usr/local/lib/python3.4/dist-packages/mdt/model_fitting.py", line 176, in run
    _, maps = self._run(self._model, self._recalculate, self._only_recalculate_last)
  File "/usr/local/lib/python3.4/dist-packages/mdt/model_fitting.py", line 213, in _run
    _in_recursion=new_in_recursion)
  File "/usr/local/lib/python3.4/dist-packages/mdt/model_fitting.py", line 222, in _run
    apply_user_provided_initialization=not _in_recursion)
  File "/usr/local/lib/python3.4/dist-packages/mdt/model_fitting.py", line 235, in _run_composite_model
    self._tmp_results_dir, recalculate=recalculate, cascade_names=model_names)
  File "/usr/local/lib/python3.4/dist-packages/mdt/model_fitting.py", line 305, in fit_composite_model
    return processing_strategy.process(worker)
  File "/usr/local/lib/python3.4/dist-packages/mdt/processing_strategies.py", line 69, in process
    self._process_chunk(processor, chunks)
  File "/usr/local/lib/python3.4/dist-packages/mdt/processing_strategies.py", line 114, in _process_chunk
    process()
  File "/usr/local/lib/python3.4/dist-packages/mdt/processing_strategies.py", line 111, in process
    processor.process(chunk, next_indices=next_chunk)
  File "/usr/local/lib/python3.4/dist-packages/mdt/processing_strategies.py", line 285, in process
    self._process(roi_indices, next_indices=next_indices)
  File "/usr/local/lib/python3.4/dist-packages/mdt/processing_strategies.py", line 442, in _process
    optimization_results = self._optimizer.minimize(codec_model, starting_positions)
  File "/usr/local/lib/python3.4/dist-packages/mot/cl_routines/optimizing/base.py", line 188, in minimize
    runner.run_procedure(self._get_optimizer_function(model),
  File "/usr/local/lib/python3.4/dist-packages/mot/cl_routines/optimizing/base.py", line 224, in _get_optimizer_function
    kernel_source += self._get_optimizer_cl_code(model)
  File "/usr/local/lib/python3.4/dist-packages/mot/cl_routines/optimizing/base.py", line 266, in _get_optimizer_cl_code
    kernel_source += self._get_optimization_function(model)
  File "/usr/local/lib/python3.4/dist-packages/mot/cl_routines/optimizing/powell.py", line 55, in _get_optimization_function
    powell_code = open(os.path.abspath(resource_filename('mot', 'data/opencl/powell.cl')), 'r').read()
  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 912: ordinal not in range(128)

There seems to be something weird with python3.4. Even when I just run mdt-list-devices, I get some prompt error:

mdt-list-devices 
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
/usr/lib/python3.4/importlib/_bootstrap.py:321: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  return f(*args, **kwds)
Device 0:
CPU - Intel(R) Xeon(R) CPU           E5607  @ 2.27GHz (Intel(R) OpenCL)

Do you know what may be going on?

Thank you in advance,

Giorgia Grisot

akhanf commented 6 years ago

Hi Giorgia, hmm not sure what is going on, I just tried mdt-list-devices and got the same warning, though haven't had a chance to run a proper test.. could perhaps ask @robbert-harms or post an issue in cbclab/MDT to see if he recognizes what might be going wrong..

robbert-harms commented 6 years ago

Hi @akhanf, Giorgia,

The RuntimeWarning can be safely ignored. See for example this answer on stackoverflow (https://stackoverflow.com/a/40846742):

These warnings are visible whenever you import scipy (or another package) that was compiled against an older numpy than is installed.

The other issue, "UnicodeDecodeError: 'ascii' codec can't decode byte" is related to using non-ascii characters in the software. This should be fixed in the latest MDT version. Do let me know if the error persist.

Best, Robbert