ds003454-mriqc: "numpy.linalg.LinAlgError: SVD did not converge"

[jbwexler]

For 2 of 24 subjects:

Node: mriqc_wf.funcMRIQC.ReportsWorkflow.PlotBrainmask Working directory: /scratch1/03201/jbwexler/work_dir/mriqc/ds003454_sub-18/mriqc_wf/funcMRIQC/ReportsWorkflow/_infile..scratch1..03201..jbwexler..openneuro_derivatives..derivatives..mriqc..ds003454-mriqc..sourcedata..raw..sub-18..func..sub-18_task-rapm_run-01_bold.nii/PlotBrainmask

Node inputs:

colors = ['r'] cut_coords = 10 display_mode = z in_contours = in_file = levels = [0.5] out_file = bmask saturate = False vmax = vmin =

Traceback (most recent call last): File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node result["result"] = node.run(updatehash=updatehash) File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run result = self._run_interface(execute=True) File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface return self._run_command(execute) File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command raise NodeExecutionError( nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node PlotBrainmask.

Traceback (most recent call last): File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run runtime = self._run_interface(runtime) File "/opt/conda/lib/python3.9/site-packages/mriqc/interfaces/viz.py", line 94, in _run_interface plot_segmentation( File "/opt/conda/lib/python3.9/site-packages/mriqc/viz/utils.py", line 578, in plot_segmentation disp = plot_anat( File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 473, in plot_anat display = plot_img(anat_img, cut_coords=cut_coords, File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 264, in plot_img display = _plot_img_with_bg( File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 185, in _plot_img_with_bg display = display_factory(display_mode)( File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/displays/_slicers.py", line 125, in init_with_figure cut_coords = cls.find_cut_coords(img, threshold, cut_coords) File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/displays/_slicers.py", line 1318, in find_cut_coords cut_coords = find_cut_slices(img, File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/find_cuts.py", line 281, in find_cut_slices img = reorder_img(img, resample='nearest') File "/opt/conda/lib/python3.9/site-packages/nilearn/image/resampling.py", line 766, in reorder_img return new_img_like(img, data, affine) File "/opt/conda/lib/python3.9/site-packages/nilearn/image/image.py", line 756, in new_img_like return klass(data, affine, header=header) File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1758, in init super(Nifti1Pair, self).init(dataobj, File "/opt/conda/lib/python3.9/site-packages/nibabel/analyze.py", line 918, in init super(AnalyzeImage, self).init( File "/opt/conda/lib/python3.9/site-packages/nibabel/spatialimages.py", line 469, in init self.update_header() File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 2034, in update_header super(Nifti1Image, self).update_header() File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1797, in update_header super(Nifti1Pair, self).update_header() File "/opt/conda/lib/python3.9/site-packages/nibabel/spatialimages.py", line 503, in update_header self._affine2header() File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1807, in _affine2header hdr.set_qform(self._affine, code='unknown') File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1024, in set_qform P, S, Qs = npl.svd(R) File "<__array_function__ internals>", line 180, in svd File "/opt/conda/lib/python3.9/site-packages/numpy/linalg/linalg.py", line 1648, in svd u, s, vh = gufunc(a, signature=signature, extobj=extobj) File "/opt/conda/lib/python3.9/site-packages/numpy/linalg/linalg.py", line 97, in _raise_linalgerror_svd_nonconvergence raise LinAlgError("SVD did not converge") numpy.linalg.LinAlgError: SVD did not converge

[jbwexler]

also for a few subjects in ds002994

[effigies]

Simply trying to set the qform from the sform in the image did not hit this problem. Worth re-running a subject to see if it's reliable.

[jbwexler]

I reran the two failed subjects from ds003454 and got the same errors.

[jbwexler]

Also getting this for a few subjects in ds004299. Any ideas?

[effigies]

TODO: Write patched python module to bind into container.

[effigies]

Patch: https://github.com/nipy/nibabel/pull/1227

To use:

datalad install -d . -s https://github.com/effigies/nibabel.git --branch enh/catch_svd_failure code/nibabel
SINGULARITYENV_PYTHONPATH="$PWD/code/nibabel" <reproman command>

@yarikoptic, just to verify, reproman will pass through these variables? Or do we need to do something like REPROENV_SINGULARITYENV_PYTHONPATH to make sure it gets through?

[yarikoptic]

to say the truth I don't remember -- I will need to try later on. If you have env handy -- just give it a shot and see.