search

nipreps / mriqc

Automated Quality Control and visual reports for Quality Assessment of structural (T1w, T2w) and functional MRI of the brain
http://mriqc.readthedocs.io
Apache License 2.0
297 stars 132 forks source link

Error when generating reports #351

Closed parekhpravesh closed 7 years ago

parekhpravesh commented 7 years ago

Hello,

When running mriqc with the "--report-dir" and "--verbose-reports" parameters such as: mriqc <path_to_BIDS_Dataset> <path_to_Output> participant --report-dir <path_to_reports_directory> --verbose-reports -<path_to_Work> the execution runs fine. The group anatomical report is generated (this dataset only has anatomical scans) and the .csv file is written. Subsequently, the following error is thrown up:

170131-04:46:32,58 mriqc INFO: Summary CSV table for the anatomical data generated (/home/JPJ/Parekh/mriqc_dst_test/mriqc_dst_25Jan2017_BIDS_Test_Output/anatMRIQC.csv) 170131-04:46:32,283 mriqc INFO: Group-anatomical report generated (/home/JPJ/Parekh/mriqc_dst_test/mriqc_dst_25Jan2017_BIDS_Test_Output/reports/anat_group.html) Traceback (most recent call last): File "/home/JPJ/Enthought/Canopy_64bit/User/bin/mriqc", line 9, in load_entry_point('mriqc==0.9.0rc1', 'console_scripts', 'mriqc')() File "/home/JPJ/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/mriqc-0.9.0rc1-py2.7.egg/mriqc/bin/mriqc_run.py", line 252, in main dataframe, out_csv = generate_csv(derivatives_dir, settings['output_dir'], qctype) File "/home/JPJ/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/mriqc-0.9.0rc1-py2.7.egg/mriqc/utils/misc.py", line 173, in generate_csv dataframe = dataframe.sort_values(by=all_id_fields) File "/home/JPJ/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pandas/core/frame.py", line 3229, in sort_values by = by[0] IndexError: list index out of range

Also, the reports directory is empty. There are no files in the Output/logs folder. Please also find attached the entire output generated during the run.

P.S: This is the workstation installation of mriqc (not the server one from which the other bug #340 has been reported)

Thanks and Regards Pravesh mriqc_test_run_output.txt

oesteban commented 7 years ago

This looks like a duplicate of #348. I'll close that one since the description here is much better.

Thanks @parekhpravesh

oesteban commented 7 years ago

Seemingly, #352 also solved this issue. Feel free to reopen in case it re-appears

parekhpravesh commented 7 years ago

On using the --verbose-reports flag, the pipeline writes the .csv file and saves the group HTML report. After that the following error is generated:

Traceback (most recent call last):

File "/home/MBIAL/Enthought/Canopy_64bit/User/bin/mriqc", line 11, in load_entry_point('mriqc==0.9.0rc1', 'console_scripts', 'mriqc')() File "/home/MBIAL/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/mriqc/bin/mriqc_run.py", line 255, in main dataframe, out_csv = generate_csv(derivatives_dir, settings['output_dir'], qctype) File "/home/MBIAL/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/mriqc/utils/misc.py", line 173, in generate_csv dataframe = dataframe.sort_values(by=all_id_fields) File "/home/MBIAL/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pandas/core/frame.py", line 3229, in sort_values by = by[0] IndexError: list index out of range

parekhpravesh commented 7 years ago

@oesteban I am unable to re-open the issue (apparently issues closed by a collaborator can not be opened by non-collaborator..something like that). Unless that the fix is actually solving the problem but something is wrong on my end?

oesteban commented 7 years ago

Could you send me the csv file and the output from mriqc?

Thanks!

parekhpravesh commented 7 years ago

Hello, Seems like I can't upload a .csv file (strange) so I have saved it as a .xls file and attached the same. I had redirected the terminal output of the run to a text file (the same is attached). This however does not contain the warnings and the error message from above (which were displayed on the terminal).

run1_server_log.txt

anatMRIQC.xlsx

Thanks!

oesteban commented 7 years ago

I'll take a look early tomorrow. Thanks very much

On Feb 2, 2017 23:16, "Pravesh Parekh" notifications@github.com wrote:

Hello, Seems like I can't upload a .csv file (strange) so I have saved it as a .xls file and attached the same. I had redirected the terminal output of the run to a text file (the same is attached). This however does not contain the warnings and the error message from above (which were displayed on the terminal).

run1_server_log.txt https://github.com/poldracklab/mriqc/files/749761/run1_server_log.txt

anatMRIQC.xlsx https://github.com/poldracklab/mriqc/files/749762/anatMRIQC.xlsx

Thanks!

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/poldracklab/mriqc/issues/351#issuecomment-277180588, or mute the thread https://github.com/notifications/unsubscribe-auth/AAkhxrnqs1bG5dg943e8gj-2l9BkRN8fks5rYtRogaJpZM4Lydt8 .

parekhpravesh commented 7 years ago

P.S: I just noticed that in the .csv the subject_id column seems to have removed the zero padding. The attachment is using the ds001 dataset where subjects are named sub-01, sub-02, and so on. However the subject_id filed simply reads as 1, 2, 3...instead of 01, 02, and 03. In an alternate run, where subjects were named sub-HS032, sub-HS034, etc. the subject_id reads HS032 and HS034 as expected.

oesteban commented 7 years ago

Hi @parekhpravesh, I don't find your error in the log file you submitted. Apparently, everything worked out ok in that run.

Regarding the zero padding trimmed in the xlsx file you sent: is it also happening in the output csv file?

parekhpravesh commented 7 years ago

Hello @oesteban , The error messages were displayed on the terminal (and hence the log files are clean). This happens on every run...the warning messages and the error messages on the terminal once the .csv is generated. Regarding the zero padding, I would only be able to confirm that in the morning now (I have left lab for the day)...sorry about that.

parekhpravesh commented 7 years ago

Greetings! Sorry about the delay in updating... 1) The subject tags are fine. I was opening the sheets in Excel and so couldn't see the leading zero (Libre Office and Notepad++ confirm that the subject id is correct).

2) The error about list index out of range persists. It turns up even when "--verbose-reports" is not passed. If you would like me to update my installation and recheck, do let me know.

Thanks!

oesteban commented 7 years ago

Hi @parekhpravesh

Could you please:

pip install git+https://github.com/oesteban/mriqc.git@fix/351

Confirm that mriqc is using version 0.9.0-rc3:

mriqc -h

And then check if the error is still happening?

Thank you

parekhpravesh commented 7 years ago

@oesteban Sure...will do so and update asap (probably by tomorrow morning; we have just scheduled a few server updates)

oesteban commented 7 years ago

Finally reproduced this myself. A fix will be coming soon.

oesteban commented 7 years ago

Got it. This is what is happening: mriqc does not deal correctly the absence of functional derivatives (a json file per input image with the calculated quality metrics).

I guess you are running mriqc only on T1w data. So everything works well (actually your csv table for the anatomical images is created, and you should also have an out/reports/anat_group.html file generated), but it breaks trying to generate those (csv table and group html file) for the functional images which do not exist.

I am fixing the group analysis level to fail with an informative warning (#372).

parekhpravesh commented 7 years ago

Hello @oesteban! Apologies, I was away. I seem to have missed all the fun while the commits and updates were happening. Indeed, I was running the tests only on the anatomical scans and that the .csv and anat-group.html are both getting created.

Thanks a lot!