search

PennLINC / xcp_d

Post-processing of fMRIPrep, NiBabies, and HCP outputs
https://xcp-d.readthedocs.io
BSD 3-Clause "New" or "Revised" License
78 stars 26 forks source link

32K anat fmriprep output crash #1305

Closed juansanchezpena closed 1 week ago

juansanchezpena commented 2 weeks ago

Summary

fmriprep 24.0.1 now outputs 32k anat surfaces in addition to the native surfaces: sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_inflated.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_midthickness.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_vinflated.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_pial.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_white.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_inflated.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_midthickness.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_vinflated.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_pial.surf.gii sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_white.surf.gii This causes error: More than one surface found was the error. When removed xcp works fine. ## Additional details

What were you trying to do? keep 32k surfaces from fmriprep in derivatives

What did you expect to happen? xcp-d would ignore 32k surface

What actually happened? More than one surface found was the error.

Reproducing the bug

tsalo commented 2 weeks ago

@juansanchezpena did you run each session through fMRIPrep separately?

juansanchezpena commented 2 weeks ago

Thanks Taylor I ran the 24.0.1 on a single subject through docker. I know previous versions never outputted he 32K anat space before. Once I removed those files xcp-d 0.9.1 finished successfully Is this clear? J

tsalo commented 2 weeks ago

XCP-D ingresses those files normally though. I have a couple of tests that include those files and are passing, so I'm guessing something went wrong.

Can you share (1) your XCP-D command and (2) the tree of your fMRIPrep derivatives for the subject you tested?

juansanchezpena commented 2 weeks ago

Sure: docker command:

docker service create --replicas 1 --reserve-cpu 8 --reserve-memory 16g --mode replicated --restart-condition none  --mount type=bind,source=$indir,destination=/input,readonly=true --mount type=bind,source=$outdir,destination=/output  --mount type=bind,source=/MRI_DATA/bin/freesurferlicense.txt,destination=/opt/freesurfer/license.txt,readonly=true  pennlinc/xcp_d:0.9.1  /input /output participant --mode none --participant-label patensasc0085  --input-type fmriprep --file-format cifti  --despike n -f 0.3 -p 36P -m y  --motion-filter-type notch  --band-stop-min 12 --band-stop-max 18 --output-type interpolated --min-coverage 0 --abcc-qc n --linc-qc y --warp-surfaces-native2std n

script:

/MRI_DATA/bin/sub-xcp_d-0.9.1  -i /MRI_DATA/mritesting/patensasc/derivatives/subfmri-arg-24.0.1 -o /MRI_DATA/mritesting/patensasc/derivatives/subfmri-arg-24.0.1/xcp-0.9 -l  /MRI_DATA/mritesting/patensasc/derivatives/subfmri-arg-24.0.1/xcp.log -a " --mode none --participant-label patensasc0085  --input-type fmriprep --file-format cifti  --despike n -f 0.3 -p 36P -m y  --motion-filter-type notch  --band-stop-min 12 --band-stop-max 18 --output-type interpolated --min-coverage 0 --abcc-qc n --linc-qc y --warp-surfaces-native2std n”

Fmriprep tree: This worked to completion:

/MRI_DATA/mritesting/patensasc/derivatives/subfmri-arg-24.0.1/sub-patensasc0085/ses-1173/anat
ls *surf.gii
sub-patensasc0085_ses-1173_hemi-L_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_midthickness.surf.gii
sub-patensasc0085_ses-1173_hemi-L_pial.surf.gii
sub-patensasc0085_ses-1173_hemi-L_space-fsLR_desc-msmsulc_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_space-fsLR_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_white.surf.gii
sub-patensasc0085_ses-1173_hemi-R_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_midthickness.surf.gii
sub-patensasc0085_ses-1173_hemi-R_pial.surf.gii
sub-patensasc0085_ses-1173_hemi-R_space-fsLR_desc-msmsulc_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_space-fsLR_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_white.surf.gii

This did not:

sub-patensasc0085_ses-1173_hemi-L_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_midthickness.surf.gii
sub-patensasc0085_ses-1173_hemi-L_pial.surf.gii
sub-patensasc0085_ses-1173_hemi-L_space-fsLR_desc-msmsulc_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_space-fsLR_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-L_white.surf.gii
sub-patensasc0085_ses-1173_hemi-R_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_midthickness.surf.gii
sub-patensasc0085_ses-1173_hemi-R_pial.surf.gii
sub-patensasc0085_ses-1173_hemi-R_space-fsLR_desc-msmsulc_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_space-fsLR_desc-reg_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_sphere.surf.gii
sub-patensasc0085_ses-1173_hemi-R_white.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_inflated.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_midthickness.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_desc-hcp_vinflated.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_pial.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_white.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_inflated.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_midthickness.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_desc-hcp_vinflated.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_pial.surf.gii
sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-R_white.surf.gii
tsalo commented 2 weeks ago

Ah, thank you. I missed that these were in fsLR space. I'll look into this.

juansanchezpena commented 2 weeks ago

Thanks for the super prompt reply

From: Taylor Salo @.> Date: Monday, October 28, 2024 at 1:13 PM To: PennLINC/xcp_d @.> Cc: Sanchez, Juan (NYSPI) @.>, Mention @.> Subject: Re: [PennLINC/xcp_d] 32K anat fmriprep output crash (Issue #1305)

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.

Ah, thank you. I missed that these were in fsLR space. I'll look into this.

— Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/xcp_d/issues/1305#issuecomment-2442170416, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARKALFFEFFDQFZ4GED63WTDZ5ZWEFAVCNFSM6AAAAABQX4AMFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBSGE3TANBRGY. You are receiving this because you were mentioned.Message ID: @.***>

tsalo commented 1 week ago

Can you share the log file from your failing run?

juansanchezpena commented 1 week ago

I just put back the 32k_FSlr and re ran our wrapper script with the same command that I posted: @.:/MRI_DATA/mritesting/patensasc/derivatives/subfmri-arg-24.0.1$ logs wqfbiv3t48o1 @. | 241031-15:24:24,170 nipype.workflow IMPORTANT: @. | Running XCP-D version 0.9.1 @. | 241031-15:24:24,201 nipype.workflow WARNING: @. | Previous output generated by version 0+unknown found. @. | 241031-15:24:27,981 nipype.workflow IMPORTANT: @. | Building XCP-D's workflow: @. | * Preprocessing derivatives path: /input. @. | Participant list: ['patensasc0085']. **@. | Run identifier: 20241031-152246_c65207df-dedd-42d6-a3a6-cb51e63e4fd7. **@. | 241031-15:24:29,630 nipype.utils WARNING: @. | Both T1w and T2w found. Checking for T1w-space T2w. @. | 241031-15:24:29,702 nipype.utils WARNING: @. | No T1w-space T2w found. Checking for T2w-space T1w. @. | 241031-15:24:29,768 nipype.utils WARNING: @. | No T2w-space T1w found. Attempting T2w-only processing. @. | 241031-15:24:29,836 nipype.utils WARNING: @. | T2w-to-template transform not found. Attempting T1w-only processing. @. | 241031-15:24:30,354 nipype.utils IMPORTANT: @. | Collected data: @. | anat_brainmask: /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_space-MNI152NLin2009cAsym_res-2_desc-brain_mask.nii.gz @. | anat_to_template_xfm: /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5 @. | bold: @. | - /input/sub-patensasc0085/ses-1173/func/sub-patensasc0085_ses-1173_task-gbu_space-fsLR_den-91k_bold.dtseries.nii @. | - /input/sub-patensasc0085/ses-1173/func/sub-patensasc0085_ses-1173_task-rest_run-01_space-fsLR_den-91k_bold.dtseries.nii @. | - /input/sub-patensasc0085/ses-1173/func/sub-patensasc0085_ses-1173_task-rest_run-02_space-fsLR_den-91k_bold.dtseries.nii @. | - /input/sub-patensasc0085/ses-1173/func/sub-patensasc0085_ses-1173_task-rest_run-03_space-fsLR_den-91k_bold.dtseries.nii @. | - /input/sub-patensasc0085/ses-1173/func/sub-patensasc0085_ses-1173_task-rest_run-04_space-fsLR_den-91k_bold.dtseries.nii @. | t1w: /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_desc-preproc_T1w.nii.gz @. | t2w: /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_desc-preproc_T2w.nii.gz @. | template_to_anat_xfm: /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5 @. | @. | Process Process-2: @. | Traceback (most recent call last): @. | File "/usr/local/miniconda/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap @. | self.run() @. | File "/usr/local/miniconda/lib/python3.10/multiprocessing/process.py", line 108, in run @. | self._target(self._args, self._kwargs) @. | File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/cli/workflow.py", line 99, in build_workflow @. | retval["workflow"] = init_xcpd_wf() @. | File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/workflows/base.py", line 78, in init_xcpd_wf @. | single_subject_wf = init_single_subject_wf(subject_id) @. | File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/workflows/base.py", line 135, in init_single_subject_wf @. | mesh_available, standard_space_mesh, software, mesh_files = collect_mesh_data( @. | File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/utils/bids.py", line 446, in collect_mesh_data @. | raise ValueError( @. | ValueError: More than one surface found. @. | Surfaces found: @. | /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_hemi-L_pial.surf.gii @. | /input/sub-patensasc0085/ses-1173/anat/sub-patensasc0085_ses-1173_space-fsLR_den-32k_hemi-L_pial.surf.gii @. | Query: {'subject': 'patensasc0085', 'datatype': 'anat', 'desc': None, 'extension': '.surf.gii', 'hemi': 'L', 'suffix': 'pial'} “

From: Taylor Salo @.> Date: Thursday, October 31, 2024 at 10:59 AM To: PennLINC/xcp_d @.> Cc: Sanchez, Juan (NYSPI) @.>, Mention @.> Subject: Re: [PennLINC/xcp_d] 32K anat fmriprep output crash (Issue #1305)

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.

Can you share the log file from your failing run?

— Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/xcp_d/issues/1305#issuecomment-2450102478, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARKALFE6QFKAGBDCS4WMOJLZ6JAV5AVCNFSM6AAAAABQX4AMFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJQGEYDENBXHA. You are receiving this because you were mentioned.Message ID: @.***>

tsalo commented 1 week ago

It should be fixed now. The fix should push to DockerHub under the unstable tag in a couple of hours. If you could pull the updated version and try it out on your data to make sure the fixed worked that would be great.

juansanchezpena commented 1 week ago

Thanks for this! I will pull Monday and get back to you with results. I am assuming it will be the latest v on git J

From: Taylor Salo @.> Date: Thursday, October 31, 2024 at 1:17 PM To: PennLINC/xcp_d @.> Cc: Sanchez, Juan (NYSPI) @.>, Mention @.> Subject: Re: [PennLINC/xcp_d] 32K anat fmriprep output crash (Issue #1305)

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.

It should be fixed now. The fix should push to DockerHub under the unstable tag in a couple of hours. If you could pull the updated version and try it out on your data to make sure the fixed worked that would be great.

— Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/xcp_d/issues/1305#issuecomment-2450413499, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARKALFG3Z2X54GJ26NYUZ63Z6JQZZAVCNFSM6AAAAABQX4AMFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJQGQYTGNBZHE. You are receiving this because you were mentioned.Message ID: @.***>

juansanchezpena commented 1 week ago

Hey Taylor Wanted to let you know the fix worked! Thank you so much. Can you let me know if 0.10.0rc3https://github.com/PennLINC/xcp_d/releases/tag/0.10.0rc3 has the update? J

From: Taylor Salo @.> Date: Thursday, October 31, 2024 at 1:17 PM To: PennLINC/xcp_d @.> Cc: Sanchez, Juan (NYSPI) @.>, Mention @.> Subject: Re: [PennLINC/xcp_d] 32K anat fmriprep output crash (Issue #1305)

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.

It should be fixed now. The fix should push to DockerHub under the unstable tag in a couple of hours. If you could pull the updated version and try it out on your data to make sure the fixed worked that would be great.

— Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/xcp_d/issues/1305#issuecomment-2450413499, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARKALFG3Z2X54GJ26NYUZ63Z6JQZZAVCNFSM6AAAAABQX4AMFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJQGQYTGNBZHE. You are receiving this because you were mentioned.Message ID: @.***>

tsalo commented 1 week ago

Yes, it should be in 0.10.0rc3.

zhitao-guo commented 8 hours ago

fmriprep 24.0.1 now outputs 32k anat surfaces in addition to the native surfaces:

Hello juan, thanks for sharing the important infomation. I have processed my data using the official fmriprep with version 24.1.1 (docker pull nipreps/fmriprep:24.1.1). But I noticed that the native fsLR32k mesh files were not generated in the anat folder. Would you kindly be able to share the command you used for this? Thanks a lot!

zhitao-guo commented 8 hours ago

Yes, it should be in 0.10.0rc3.

Hello Taylor, I would kindly ask the time of the next formal release of XCP-D. I’m looking forward to it.

juansanchezpena commented 8 hours ago

--output-spaces T1w MNI152NLin2009cAsym:res-2 --cifti-output --project-goodvoxels

Get Outlook for iOShttps://aka.ms/o0ukef

From: Zhitao Guo @.> Sent: Wednesday, November 13, 2024 11:17:10 AM To: PennLINC/xcp_d @.> Cc: Sanchez, Juan (NYSPI) @.>; Mention @.> Subject: Re: [PennLINC/xcp_d] 32K anat fmriprep output crash (Issue #1305)

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.

Yes, it should be in 0.10.0rc3.

Hello Taylor, I would kindly ask the time of the next formal release of XCP-D. I’m looking forward to it.

— Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/xcp_d/issues/1305#issuecomment-2474084631, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARKALFGPYKNW3TELMQJUTS32AN3QNAVCNFSM6AAAAABQX4AMFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZUGA4DINRTGE. You are receiving this because you were mentioned.Message ID: @.***>

zhitao-guo commented 7 hours ago

--output-spaces T1w MNI152NLin2009cAsym:res-2 --cifti-output --project-goodvoxels

Thanks. I will hava a try.