FreeSurfer License flag only works if license is bound to /opt/freesurfer/license.txt within container

[tjhendrickson]

Summary

FreeSurfer License flag only works if license is bound to /opt/freesurfer/license.txt within container

Additional details

• xcp_d version: 0.6.1
• Docker version: 0.6.1
• Singularity version:

What were you trying to do?

Run XCP-D from beginning to end

What did you expect to happen?

For XCP-D to run to completion

What actually happened?

Received a long stacktrace, centered around the FreeSurfer license not being found

Node: xcpd_wf.single_subject_958739_wf.postprocess_surfaces_wf.warp_surfaces_to_template_wf.lh_apply_transforms_wf.sphere_to_surf_gii
Working directory: /scratch.global/msi_cbrain_hbcd_mcgill/cbrain_tasks/266/57/45/elee-XCPD-T2665745/work/xcpd_wf/single_subject_958739_wf/postprocess_surfaces_wf/warp_surfaces_to_template_wf/lh_apply_transforms_wf/sphere_to_surf_gii
Node inputs:
annot_file = <undefined>
args = <undefined>
dataarray_num = <undefined>
environ = {'SUBJECTS_DIR': '/opt/freesurfer/subjects'}
functional_file = <undefined>
in_file = <undefined>
label_file = <undefined>
labelstats_outfile = <undefined>
normal = <undefined>
origname = <undefined>
out_datatype = gii
out_file = <undefined>
parcstats_file = <undefined>
patch = <undefined>
rescale = <undefined>
scalarcurv_file = <undefined>
scale = <undefined>
subjects_dir = /opt/freesurfer/subjects
talairachxfm_subjid = <undefined>
to_scanner = <undefined>
to_tkr = <undefined>
vertex = <undefined>
xyz_ascii = <undefined>
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node sphere_to_surf_gii.
Cmdline:
    mris_convert /scratch.global/msi_cbrain_hbcd_mcgill/cbrain_tasks/266/57/45/elee-XCPD-T2665745/fmri/sourcedata/mcribs/sub-958739/freesurfer/sub-958739/surf/lh.sphere.reg2 /scratch.global/msi_cbrain_hbcd_mcgill/cbrain_tasks/266/57/45/elee-XCPD-T2665745/work/xcpd_wf/single_subject_958739_wf/postprocess_surfaces_wf/warp_surfaces_to_template_wf/lh_apply_transforms_wf/sphere_to_surf_gii/lh.sphere.reg2_converted.gii
Stdout:
Stderr:
    --------------------------------------------------------------------------
    ERROR: FreeSurfer license file /opt/freesurfer/license.txt not found.
      If you are outside the NMR-Martinos Center,
      go to http://surfer.nmr.mgh.harvard.edu/registration.html to 
      get a valid license file (it's free).
      If you are inside the NMR-Martinos Center,
      make sure to source the standard environment.
      A path to an alternative license file can also be
      specified with the FS_LICENSE environmental variable.
    --------------------------------------------------------------------------
Traceback:
    Traceback (most recent call last):
      File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
        setattr(outputs, key, val)
      File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
        value = super(File, self).validate(objekt, name, value, return_pathlike=True)
      File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
        self.error(objekt, name, str(value))
      File "/usr/local/miniconda/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
        raise TraitError(
    traits.trait_errors.TraitError: The 'converted' trait of a MRIsConvertOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/scratch.global/msi_cbrain_hbcd_mcgill/cbrain_tasks/266/57/45/elee-XCPD-T2665745/work/xcpd_wf/single_subject_958739_wf/postprocess_surfaces_wf/warp_surfaces_to_template_wf/lh_apply_transforms_wf/sphere_to_surf_gii/lh.sphere.reg2_converted.gii' <class 'str'> was specified.
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
      File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 400, in run
        outputs = self.aggregate_outputs(runtime)
      File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
        raise FileNotFoundError(msg)
    FileNotFoundError: No such file or directory '/scratch.global/msi_cbrain_hbcd_mcgill/cbrain_tasks/266/57/45/elee-XCPD-T2665745/work/xcpd_wf/single_subject_958739_wf/postprocess_surfaces_wf/warp_surfaces_to_template_wf/lh_apply_transforms_wf/sphere_to_surf_gii/lh.sphere.reg2_converted.gii' for output 'converted' of a MRIsConvert interface

Reproducing the bug

Binding the freesurfer license file within the container anywhere but /opt/freesurfer/license.txt

[tsalo]

Do you have an example docker call that failed?

[tjhendrickson]

singularity run -B $input:/input \
-B $output:/output \
-B $work:/work \
-B $fs_license:/license.txt \
$container_path /input /output participant \
--input-type nibabies \
--cifti \
--despike \
--dcan-qc \
-w /work \
--omp-nthreads 3 \
--warp-surfaces-native2std \
-f 0.3 \
--head_radius auto \
--combineruns \
-vv \
--fs-license-file /license.txt

You will find the stacktrace that I laid out above in the standard out, not standard error

[mattcieslak]

I think a --cleanenv would clear this up

[tjhendrickson]

Good thought, I went ahead and tried this, but unfortunately received the same error.

[tsalo]

@tjhendrickson can you do echo $FS_LICENSE? I wonder if XCP-D somehow has a problem when the FS_LICENSE environment variable and --fs-license-file values don't match.

[mattcieslak]

it looks like $fs_license (lowercase) is being bound to /license.txt

[tjhendrickson]

Yes, $fs_license is just a variable representing a file path: fs_license=/home/faird/shared/projects/HBCD_MIDB_IG/xcpd/testing_feb27_24/license.txt

Of note, everything works just fine when $fs_license is being bound to /opt/freesurfer/license.txt.

Best,

-Tim

[tsalo]

I'm going to try to reproduce the issue on CUBIC later today.

[tsalo]

I'm not able to reproduce your error, but I have one more idea. Can you share your full error log from that last attempt? You can DM it to me on Slack if you want.

[tsalo]

Is it possible the license file you used (/home/faird/shared/projects/HBCD_MIDB_IG/xcpd/testing_feb27_24/license.txt) is a symlink?

[tjhendrickson]

I was able to resolve this by leveraging the --env apptainer/singularity flag, specifically --env FS_LICENSE=/license.txt