search

nipreps / mriqc

Automated Quality Control and visual reports for Quality Assessment of structural (T1w, T2w) and functional MRI of the brain
http://mriqc.readthedocs.io
Apache License 2.0
299 stars 132 forks source link

Numeric overflow when computing drift on int16 DWI (that has slope) #1315

Closed psadil closed 3 months ago

psadil commented 3 months ago

What happened?

I'm processing INT16 DWI images that are stored with a slope

❯ fslinfo $dwi
data_type   INT16
dim1        144
dim2        144
dim3        81
dim4        102
datatype    4
pixdim1     1.666667
pixdim2     1.666667
pixdim3     1.700000
pixdim4     8.093593
cal_max     0.000000
cal_min     0.000000
file_type   NIFTI-1+

❯ fslhd $dwi | grep scl
scl_slope   45.075214
scl_inter   0.000000

The intensity values are quite high. Note that the following max is after application of the slope, and that it happens to be larger than what can be represented by int16

❯ fslstats $dwi -R
0.000000 145683.093750

The output of the drift node ends up overflowing

❯ fslstats work/mriqc_wf/dwiMRIQC/_in_file_[...]/drift/sub-travel1_ses-UC_dwi_nodriftfull.nii.gz -R
-32751.000000 32763.000000

For that output, the int16 dtype was preserved, but not the slope

❯ fslhd work/mriqc_wf/dwiMRIQC/_in_file_[...]/drift/sub-travel1_ses-UC_dwi_nodriftfull.nii.gz | grep -E "scl|type"
data_type   INT16
datatype    4
scl_slope   1.000000
scl_inter   0.000000
file_type   NIFTI-1+

Seems like the image was casted down to int16 without use of the slope, around here https://github.com/nipreps/mriqc/blob/a6a2ba8ac88cc1fd1b1e544acf402d7a52642100/mriqc/interfaces/diffusion.py#L660-L671

The full workflow ends up crashing with messages about invalid quality measures, which were presumably caused by the overflow and negative values.

Two possible solutions come to mind

What command did you use?

docker run --rm -it -v $PWD:$PWD nipreps/mriqc:24.0.0 -w $PWD/work -vvv $PWD/bids/UCtravel1 mriqc participant

What version of the software are you running?

24.0.0

How are you running this software?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

2024-08-08 12:47:12 | INFO     | nipype.workflow  | [Node] Finished "measures", elapsed time 17.344293s.
2024-08-08 12:47:13 | INFO     | nipype.workflow  | [Node] Setting-up "mriqc_wf.dwiMRIQC.ComputeIQMs.datasink" in "/Users/psadil/Library/CloudStorage/OneDrive-JohnsHopkins/data/traveling/work/mriqc_wf/dwiMRIQC/ComputeIQMs/_in_file_..Users..psadil..Library..CloudStorage..OneDrive-JohnsHopkins..data..traveling..bids..UCtravel1..sub-travel1..ses-UC..dwi..sub-travel1_ses-UC_dwi.nii.gz/datasink".
2024-08-08 12:47:13 | INFO     | nipype.workflow  | [Node] Executing "datasink" <mriqc.interfaces.bids.IQMFileSink>
2024-08-08 12:47:13 | INFO     | nipype.workflow  | [Node] Finished "datasink", elapsed time 0.000875s.
2024-08-08 12:47:13 | WARNING  | nipype.workflow  | Storing result file without outputs
2024-08-08 12:47:13 | WARNING  | nipype.workflow  | [Node] Error on "mriqc_wf.dwiMRIQC.ComputeIQMs.datasink" (/Users/psadil/Library/CloudStorage/OneDrive-JohnsHopkins/data/traveling/work/mriqc_wf/dwiMRIQC/ComputeIQMs/_in_file_..Users..psadil..Library..CloudStorage..OneDrive-JohnsHopkins..data..traveling..bids..UCtravel1..sub-travel1..ses-UC..dwi..sub-travel1_ses-UC_dwi.nii.gz/datasink)
Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 579, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node datasink.

Traceback:
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/mriqc/interfaces/bids.py", line 188, in _run_interface
        json.dumps(
      File "/opt/conda/lib/python3.11/site-packages/simplejson/__init__.py", line 395, in dumps
        **kw).encode(obj)
              ^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 300, in encode
        chunks = list(chunks)
                 ^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 714, in _iterencode
        for chunk in _iterencode_dict(o, _current_indent_level):
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 645, in _iterencode_dict
        yield _floatstr(value)
              ^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 351, in floatstr
        raise ValueError(
    ValueError: Out of range float values are not JSON compliant: nan

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/mriqc", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/mriqc/cli/run.py", line 178, in main
    mriqc_wf.run(**_plugin)
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 214, in run
    self._send_procs_to_workers(updatehash=updatehash, graph=graph)
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 582, in _send_procs_to_workers
    self._clean_queue(
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 259, in _clean_queue
    raise RuntimeError(''.join(result['traceback']))
RuntimeError: Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 579, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node datasink.

Traceback:
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/mriqc/interfaces/bids.py", line 188, in _run_interface
        json.dumps(
      File "/opt/conda/lib/python3.11/site-packages/simplejson/__init__.py", line 395, in dumps
        **kw).encode(obj)
              ^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 300, in encode
        chunks = list(chunks)
                 ^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 714, in _iterencode
        for chunk in _iterencode_dict(o, _current_indent_level):
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 645, in _iterencode_dict
        yield _floatstr(value)
              ^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 351, in floatstr
        raise ValueError(
    ValueError: Out of range float values are not JSON compliant: nan

Additional information / screenshots

Possible duplicate of #1302 -- noticed this bug while trying to track down the source of that error, which was that some efcs were 'nan'