search

robbert-harms / MDT

Microstructure Diffusion Toolbox
GNU Lesser General Public License v3.0
50 stars 18 forks source link

`get_data` and `np.bool`, possibly others, trigger deprecation errors from NiBabel as of nibabel>=5.0.0 and numpy>=1.24.0 #57

Open kimjetwav opened 1 year ago

kimjetwav commented 1 year ago

Hey, we've encountered an incompatibility in using MDT version 1.2.7 with NiBabel versions greater than or equal to 5.0.0, in that the former utilises a deprecated interface from the latter: the get_data interface, which is now deprecated in favour of get_fdata.

Until version 5.0.0 of nibabel this would have merely triggered a deprecation warning, but given that MDT does not pin its required version of nibabel to below 5.0.0 and NiBabel is now at 5.1.0, any fresh virtual env installation of MDT will pull an incompatible nibabel with it.

Below is a log of one such error:

(venv) [redacted]@[redacted]:~$ cd /[redacted]/MDT_batchfit/data
(venv) [redacted]@[redacted]:/[redacted]/MDT_batchfit/data$ mdt-batch-fit . NODDI
[2023-11-19 17:09:04,735] [INFO] [mdt] [batch_fit] - Using MDT version 1.2.7
[2023-11-19 17:09:04,735] [INFO] [mdt] [batch_fit] - Using batch profile: DirPerSubject
[2023-11-19 17:09:04,881] [INFO] [mdt] [batch_fit] - Fitting models: ['NODDI']
[2023-11-19 17:09:04,881] [INFO] [mdt] [batch_fit] - Subjects found: [redacted]
[2023-11-19 17:09:04,882] [INFO] [mdt] [batch_fit] - Subjects to process: [redacted]
[2023-11-19 17:09:04,959] [INFO] [mdt.lib.processing.model_fitting] [__call__] - Going to process subject sub-[redacted], (1 of [redacted], we are at 0.00%)
[2023-11-19 17:09:05,197] [INFO] [mdt.lib.processing.model_fitting] [__call__] - Loading the data (DWI, mask and protocol) of subject sub-[redacted]
Traceback (most recent call last):
  File "/[redacted]/MDT/1.2.7/venv/bin/mdt-batch-fit", line 8, in <module>
    sys.exit(BatchFit.console_script())
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/shell_utils.py", line 47, in console_script
    cls().start(sys.argv[1:])
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/shell_utils.py", line 66, in start
    self.run(args, {})
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/cli_scripts/mdt_batch_fit.py", line 123, in run
    mdt.batch_fit(os.path.realpath(args.data_folder),
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/__init__.py", line 579, in batch_fit
    return batch_apply(data_folder, batch_fit_func, batch_profile=batch_profile, subjects_selection=subjects_selection)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/batch_utils.py", line 504, in batch_apply
    results[subject.subject_id] = f(subject)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/batch_utils.py", line 502, in f
    return func(subject)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 226, in __call__
    input_data = subject_info.get_input_data(use_gradient_deviations)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/batch_utils.py", line 240, in get_input_data
    return load_input_data(self._dwi_fname,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/input_data.py", line 791, in load_input_data
    mask = load_brain_mask(mask)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/utils.py", line 860, in load_brain_mask
    mask = _load_data() > 0
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/utils.py", line 851, in _load_data
    return load_nifti(data_source).get_data()
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/nifti.py", line 321, in get_data
    data = original_function(*args, **kwargs)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/nibabel/deprecator.py", line 208, in deprecated_func
    raise exception(message)
nibabel.deprecator.ExpiredDeprecationError: get_data() is deprecated in favor of get_fdata(), which has a more predictable return type. To obtain get_data() behavior going forward, use numpy.asanyarray(img.dataobj).

* deprecated from version: 3.0
* Raises <class 'nibabel.deprecator.ExpiredDeprecationError'> as of version: 5.0

This error also occurs for the usage of np.bool in NumPy greater than or equal to version 1.24.0 as well:

/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py:391: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  x_dict.update({self._used_mask_name: np.ones(roi_indices.shape[0], dtype=np.bool)})
Traceback (most recent call last):
  File "/[redacted]/MDT/1.2.7/venv/bin/mdt-batch-fit", line 8, in <module>
    sys.exit(BatchFit.console_script())
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/shell_utils.py", line 47, in console_script
    cls().start(sys.argv[1:])
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/shell_utils.py", line 66, in start
    self.run(args, {})
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/cli_scripts/mdt_batch_fit.py", line 123, in run
    mdt.batch_fit(os.path.realpath(args.data_folder),
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/__init__.py", line 579, in batch_fit
    return batch_apply(data_folder, batch_fit_func, batch_profile=batch_profile, subjects_selection=subjects_selection)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/batch_utils.py", line 504, in batch_apply
    results[subject.subject_id] = f(subject)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/batch_utils.py", line 502, in f
    return func(subject)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 238, in __call__
    fit_model(model, input_data, output_dir,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/__init__.py", line 189, in fit_model
    inits = get_optimization_inits(model_name, input_data, output_folder, cl_device_ind=cl_device_ind,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/__init__.py", line 97, in get_optimization_inits
    return get_optimization_inits(model_name, input_data, output_folder, cl_device_ind=cl_device_ind,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 177, in get_optimization_inits
    return get_init_data(model_name)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 115, in get_init_data
    fit_results = get_model_fit('BallStick_r1')
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 84, in get_model_fit
    results = fit_model(model_name, input_data, output_folder, recalculate=False, use_cascaded_inits=False,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/__init__.py", line 204, in fit_model
    fit_composite_model(model_instance, input_data, output_folder, method,
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 306, in fit_composite_model
    return processing_strategy.process(worker)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/processing_strategies.py", line 106, in process
    self._process_chunk(processor, chunks)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/processing_strategies.py", line 151, in _process_chunk
    process()
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/processing_strategies.py", line 148, in process
    processor.process(chunk, next_indices=next_chunk)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/processing_strategies.py", line 275, in process
    self._process(roi_indices, next_indices=next_indices)
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/mdt/lib/processing/model_fitting.py", line 391, in _process
    x_dict.update({self._used_mask_name: np.ones(roi_indices.shape[0], dtype=np.bool)})
  File "/[redacted]/MDT/1.2.7/venv/lib/python3.10/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?

This issue can be temporarily bandaged by specifying nibabel==4.0.2 and numpy==1.23.5 when setting up a virtual env, but of course that's just a quick fix on our end.

robbert-harms commented 1 year ago

Hi kimjetwav,

Thank you for this extensive report. This is sometimes called "code rot", the interesting fact that even if you write bug-free code (or so you hope), the world around your software changes and it suddenly no longer works. At the moment I do not have time at all to focus on MDT. Could you perhaps create a pull request?

Best, Robbert