subject html file doesn't have any visual reports like carpet plots!!

[M31glx]

Just wondering for getting visual reports, do I need to use a specific command?

[mattcieslak]

The visual reports are automatically generated at the end. There may have been a failure during the running of the pipeline, have you verified that qsiprep finished successfully?

[M31glx]

Thank you very much for your quick reply.

It certainly does preprocessing steps but doesn't give me any confirmation for finishing the process. Here is the message I got in the terminal:

210629-20:43:52,155 nipype.workflow INFO: [Node] "_mni_tpms2" found cached. 210629-20:43:52,157 nipype.workflow INFO: [Node] Finished "qsiprep_wf.single_subject_02_wf.anat_preproc_wf.mni_tpms". 210629-20:44:45,906 nipype.workflow INFO: [Node] Finished "qsiprep_wf.single_subject_02_wf.anat_preproc_wf.mni_mask". 210629-20:44:51,666 nipype.workflow INFO: [Node] Finished "qsiprep_wf.single_subject_02_wf.anat_preproc_wf.mni_seg". Sentry is attempting to send 1 pending error messages Waiting up to 2 seconds Press Ctrl-C to quit

What kind of verification do I need to be expected to receive? Because this is the only thing i see at the end of the process. However, I get subject HTML (without visual reports) and other generated files in /dwi and /anat directories. Also, i don't get anything in the /figure directory? is it normal?

Thank you very much in advance.

Best regards, Ashkan

[mattcieslak]

Based on

Sentry is attempting to send 1 pending error messages

An error must have occurred during processing. You can add --stop-on-first-crash next time you run the command to see where it's crashing or you can look for the error log in the logs directory

[M31glx]

Thanks for the quick reply.

I actually added the --stop-on-first-crash and also --verbose but still, no log file is generated in the /log directory. Here is the message again:

[Node] Finished "qsiprep_wf.single_subject_02_wf.anat_preproc_wf.anat_derivatives_wf.ds_mni_seg". 210630-03:12:52,966 nipype.workflow INFO: [Job 122] Completed (qsiprep_wf.single_subject_02_wf.anat_preproc_wf.anat_derivatives_wf.ds_mni_seg). Sentry is attempting to send 1 pending error messages Waiting up to 2 seconds Press Ctrl-C to quit

I'm using qsiprep-docker version 0.13.1 on ubuntu 18.04 . Any recommendation?

Thank you, Ashkan

[mattcieslak]

I can't tell what's happening. The best set of steps would be to

1. ensure your data is valid using the bids-validator
2. use two --verbose flags

What are the contents of your BIDS directory?

[M31glx]

The dataset is BIDS valid as below:

2363 Files, Resting-state 64 - Subjects T1w
1 - Session magnitude1
magnitude2
phase2
bold
dwi
sbref
fieldmap

I added two --verbose but still got the same error:

[Job 122] Completed (qsiprep_wf.single_subject_02_wf.anat_preproc_wf.anat_derivatives_wf.ds_mni_seg). Sentry is attempting to send 1 pending error messages Waiting up to 2 seconds Press Ctrl-C to quit

[mattcieslak]

The two verbose flags should increase the amount of text printed in the terminal, the error will still occur but it will be noisier. There will be something earlier in the terminal output that should say something about an error. Could you post the whole terminal output and the command you ran?

[M31glx]

Here is the log generated in the terminal:

log.txt

[mattcieslak]

I don't see any errors or problems in the log. What does your ~/Derivatives/qsiprep directory look like?

[jakepalmer]

I'm also getting this. Everything seems to run without error with all output images produced. I also get the "Sentry is attempting to send 1 pending error messages" at the end. There is a html report produced that only includes the methods with no visual reports and also says no errors to report.

Here's the command I ran:

qsiprep-docker ${PWD}/bids_input ${PWD}/bids_derivs participant \
    --participant_label 0009 \
    --output-resolution 1.25 \
    --use-syn-sdc \
    --force-syn \
    --fs-license-file ../code/license.txt \
    --stop-on-first-crash \
    --verbose \
    --verbose

And here are the outputs generated during processing, the input directory and qsiprep output directory contents: bids_input.txt log.txt qsiprep_contents.txt

Thanks!

[mattcieslak]

Thanks for reporting this @jakepalmer and @ashkan-alvand. Would either of you be willing to share your data so I can debug? I'm not able to replicate this in my testing

[M31glx]

Absolutely. here is the link to one of the data:

https://drive.google.com/drive/folders/1h_CD_wAoMuwtgNkjjI5AhH0MZIPyDaDj?usp=sharing

[mattcieslak]

@ashkan-alvand thanks for posting these! I downloaded them and will give this a try locally. What OS are you using to run docker?

[mattcieslak]

I think this might have something to do with not supplying a value for -w in the qsiprep call. Could you try re-running the command with specifying a working directory using -w?

[M31glx]

Thanks, Matt for following this up. It's now all good. The final report includes all those visual reports. -w was the solution.

[mattcieslak]

I'm going to reopen this to remind myself to fix this bug. You shouldn't have to specify a working directory unless you want to

[xu-boyan]

I think this issue can be fixed if the reportlets_dir is converted to absolute path.

https://github.com/PennLINC/qsiprep/blob/master/qsiprep/workflows/base.py#L227

from: reportlets_dir = os.path.join(work_dir, 'reportlets') to: reportlets_dir = os.path.join(os.path.abspath(work_dir), 'reportlets')

[xu-boyan]

The inputs.base_directory of a ds_report node is the reportlets_dir. The out_path of this node starts with this reportlets_dir (https://github.com/PennLINC/qsiprep/blob/master/qsiprep/interfaces/bids.py#L343).

However, if it is not an absolute path, this out_path will be created in the working_dir (line 345), not the work_dir/reportlets as expected (https://github.com/PennLINC/qsiprep/blob/master/qsiprep/viz/reports.py#L316).

[xu-boyan]

The out_path of a ds_report node when the value for -w is a relative path (work4 in this example).

[xu-boyan]

Hope this helps

[aghayoor]

Hi @mattcieslak , I have the same issue as reported here. Here is the qsiprep command line that was used:

qsiprep-docker $datadir/BIDS/ $outdir participant --recon_spec dsi_studio_gqi --participant-label 01 --output-resolution 2 --fs-license-file ./fs_license.txt --verbose --verbose -- stop-on-first-crash

The tool ran with no crash, and all outputs were successfully written to the \qsiprep and \qsirecon directories. I have attached output screen log and the list of files in the output derivatives directory.

However, the output html only includes the methods and references with no error reported. A screenshot is attached here.

I have also attached the log of my input directory, which has been passed successfully through the BIDS validator tool.

inputBIDS_log.txt outputdir_log.txt screenlog.txt

[aghayoor]

@mattcieslak , I also tried to run the tool with -w flag:

qsiprep-docker $datadir/BIDS/ $outdir participant -w $workdir --recon_spec dsi_studio_gqi --participant-label 01 --output-resolution 2 --fs-license-file ./fs_license.txt --verbose --verbose --stop-on-first-crash

However, it crashed shortly after the run. The screen log is attached here.

output_log_with_work_dir.txt

[mattcieslak]

@aghayoor your issue is due to not specifying --recon-input, which needs to point to the output directory from the preprocessing qsiprep run

[aghayoor]

@mattcieslak , which of these issues are caused by not specifying recon-input? The QC html file or the -w error?

Both issues persist even when I don't run the reconstruction workflow.

I have attached the screen log based on the following command line:

qsiprep-docker $datadir/BIDS/ $outdir participant -w $workdir --participant-label 01 --output-resolution 2 --fs-license-file ./fs_license.txt --verbose --verbose --stop-on-first-crash

screenlog_with_workdir_norecon.txt

[mattcieslak]

This has something to do with qslite3, which I've never seen before. Does your data pass the bids validator? If so, you might need to open an issue on PyBIDS

Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/miniconda/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

This may help

[aghayoor]

@mattcieslak , Thanks! I had the work directory next to my output derivatives directory. I changed the work directory to be in my home directory, and this avoided the crash! However, this does fully makes sense to me. I had write/read permission to the work directory the same as the output directory, so how the work directory is treated differently?

[dkp]

I am encountering the same issue: visual reports are not being generated. I'm using a BIDs dataset that works fine for all the other BIDS tools I tested (fmriprep, MRIQC, MRtrix3_connectome)... the command I ran was:

docker run --rm -it 
-v /Users/dpat/license.txt:/opt/freesurfer/license.txt:ro 
-v /Volumes/Main/Working/QSIPrep/data:/data:ro 
-v /Volumes/Main/Working/QSIPrep/derivatives:/qsiprep-output:ro 
-v /Volumes/Main/Working/QSIPrep/derivatives:/out pennbbl/qsiprep:0.14.2 /data /out participant --recon-input /qsiprep-output --recon-spec mrtrix_singleshell_ss3t --output-resolution 1.3

[mattcieslak]

Hi @dkp this bug should be fixed in 0.14.2, but it looks like you're still encountering it - could you please try specifying a working directory in your command?

Something like:

docker run --rm -it \
-v /Users/dpat/license.txt:/opt/freesurfer/license.txt:ro \
-v /Volumes/Main/Working/QSIPrep/data:/data:ro \
-v /Volumes/Main/Working/QSIPrep/derivatives:/qsiprep-output:ro \
-v /Volumes/Main/Working/QSIPrep/work:/work \
-v /Volumes/Main/Working/QSIPrep/derivatives:/out pennbbl/qsiprep:0.14.2 \
    /data /out participant \
    --recon-input /qsiprep-output \
    --recon-spec mrtrix_singleshell_ss3t \
    --output-resolution 1.3 \
    -w /work -v -v

[dkp]

That worked! What pretty pictures. Thanks so much.

-Dianne

Dianne Patterson, Ph.D Speech, Language and Hearing Sciences, Room 314 @.***

---

From: Matt Cieslak @.> Sent: Monday, September 13, 2021 7:08 AM To: PennLINC/qsiprep @.> Cc: Patterson, Dianne K - (dkp) @.>; Mention @.> Subject: [EXT]Re: [PennLINC/qsiprep] subject html file doesn't have any visual reports like carpet plots!! (#262)

External Email

Hi @dkphttps://github.com/dkp this bug should be fixed in 0.14.2, but it looks like you're still encountering it - could you please try specifying a working directory in your command?

Something like:

docker run --rm -it \ -v /Users/dpat/license.txt:/opt/freesurfer/license.txt:ro \ -v /Volumes/Main/Working/QSIPrep/data:/data:ro \ -v /Volumes/Main/Working/QSIPrep/derivatives:/qsiprep-output:ro \ -v /Volumes/Main/Working/QSIPrep/work:/work \ -v /Volumes/Main/Working/QSIPrep/derivatives:/out pennbbl/qsiprep:0.14.2 \ /data /out participant \ --recon-input /qsiprep-output \ --recon-spec mrtrix_singleshell_ss3t \ --output-resolution 1.3 \ -w /work -v -v

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/PennLINC/qsiprep/issues/262#issuecomment-918230043, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHLUQEGYIANVWV3VRAHWS3UBYAWJANCNFSM47PHZNGA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[giulia-berto]

Hello, I am encountering the same issue, i.e. the html report is only showing methods and references but no figures (the folder figures in the output is empty). This is happening with versions 0.14.2 and 0.14.3, but I had no issues with version 0.13.0RC1. My command is

time singularity exec -e \
    docker://pennbbl/qsiprep:0.14.3 \
    /usr/local/miniconda/bin/qsiprep \
    --fs-license-file license.txt \
    --output-resolution $resolution \
    --output-space $space \
    --denoise-method $denoise_method \
    --unringing-method $unringing_method \
    --hmc-transform $hmc_transform \
    --nthreads 8 \
    --verbose \
    bids $outdir participant \
    -w $WORKDIRNAME

[naveau]

Sorry for digging this out but this issue is still open so I continue on this thread. There is probably still a remaining issue regarding the reports/work-dir point.

Running qsiprep with the --work-dir option does not produce report if the working directory path is relative.

Not sure it's a fully reproducible example but here is the steps I follow : Given a BIDS valid dataset, this won't produce report (except the html method boilerplate) :

participant="participantid"
qsiprep \
    --participant-label ${participant} \
    --output-resolution 1.2 \
    --nthreads 10 \
    --omp-nthreads 10 \
    --stop-on-first-crash \
    --fs-license-file license.txt \
    --work-dir work/sub-${participant}/qsiprep/ \
    --verbose \
    BIDS/ derivatives/qsiprep/ participant

Adding $PWD to setup the workdir as a full path solve the report issue :

participant="participantid"
qsiprep \
    --participant-label ${participant} \
    --output-resolution 1.2 \
    --nthreads 10 \
    --omp-nthreads 10 \
    --stop-on-first-crash \
    --fs-license-file license.txt \
    --work-dir $PWD/work/sub-${participant}/qsiprep/ \
    --verbose \
    BIDS/ derivatives/qsiprep/ participant

I don't know where to search in the code to investigate this further but if someone as an hint I could give it a try. In the meantime, please use a fullpath for your working dir !