search

bids-apps / freesurfer

BIDS app wrapping recon-all from FreeSurfer
Apache License 2.0
41 stars 35 forks source link

QA #9

Open fliem opened 7 years ago

fliem commented 7 years ago

Is there anything I can do to revive the QA branch?

chrisgorgo commented 7 years ago

It requires using a mini X server and I could not get it to work reliably under Docker and Singularity. It's an old and clunky solution - are there any other QA oriented FreeSurfer tools out there?

fliem commented 7 years ago

Mhh. I have a vague memory of @satra heading a past brainhack project about FS QA?

satra commented 7 years ago

@fliem - this depends on whether you are willing to create something from scratch or use some of the outputs in place.

for example, the brain is generally quite symmetric - so bad reconstructions often have abs(1 - L_struct/R_struct) > some number for each aparc/a2009 structure

but the things that need QA and are hard are:

many of these should be detectable with enough training data.

fliem commented 7 years ago

Since surface plots now arrived in nilearn, and creating qc plots for freesurfer is such a drag, I suggest we revive this issue. As a starting point I have created a couple of plots in a notebook. What do you think?

chrisgorgo commented 7 years ago

This looks great - looping in @effigies who did some surface QC for fmriprep. This is how it looks:

image

alexlicohen commented 7 years ago

I also came across this: https://github.com/akeshavan/mindcontrol

Which might be work a look or forking some ideas from...

-Alex

On Mar 4, 2017, at 2:21 PM, Chris Filo Gorgolewski notifications@github.com wrote:

This looks great - looping in @effigies who did some surface QC for fmriprep. This is how it looks:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

fliem commented 7 years ago

Those look very pretty. @effigies could you please point me towards that fmriprep code. Did you do other things for qc as well (like calculating metrics...)? Thanks.

fliem commented 7 years ago

I have refined the surface and subcortical plots, borrowing heavily from nilearn/niworkflows. Next, I'll look into how to assemble them in htmls

effigies commented 7 years ago

Hi @fliem sorry that I totally missed this thread.

It looks like you found the visualization bit in niworkflows. We haven't really done any metrics at this point, though; we've reconstructed surfaces using ANTs skull-stripping instead of mri_watershed, and are now using bbregister to perform functional alignment. At present, my focus (poldracklab/fmriprep#391) is on sampling functional data to the surface and ensuring that these surfaces align properly in native T1 space, so that they can be used smoothly in non-FreeSurfer tools.

fliem commented 7 years ago

I have created first examples for single subjects and group html reports. Any suggestions?

chrisgorgo commented 7 years ago

They look great, although the subcortical segmentation seem a little misaligned.

I wonder if this would make more sense as a standalone python module and command line tool that could be reused in other projects.

On Mon, May 8, 2017 at 11:18 AM, Franz Liem notifications@github.com wrote:

I have created first examples for single subjects and group html reports https://github.com/fliem/freesurfer/tree/qc_nb/example_reports/00_qc. Any suggestions?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BIDS-Apps/freesurfer/issues/9#issuecomment-299947850, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkpxGZiyhqEt7Bs_mgsBIDothPG40Lks5r31xzgaJpZM4LUmfj .

effigies commented 7 years ago

I can't see the parcellation reports at all, either through Github or pulling locally and viewing...

But I do like @chrisfilo's idea of a separate module that would ease re-use in other projects. Since the base machinery for this is niworkflows-based, I think it'd be reasonable to pull this into niworkflows, itself. But I'd also entirely support an independent module that we then incorporate into ReconAllRPT.

fliem commented 7 years ago
alexlicohen commented 7 years ago

To view these without cloning, use rawgit.com

https://cdn.rawgit.com/fliem/freesurfer/qc_nb/example_reports/00_qc/freesurfer_qc_group_report.html

This looks pretty nice actually for scanning for large-scale FS failures. (For more detailed screening I think we'll still need the more detailed manual review and/or quantitative assessment like mindcontrol or mriqc

-Alex

Alexander Li Cohen, M.D., Ph.D. E-mail: Alexander.Cohen2@childrens.harvard.edu (Medical/Science Email) E-mail: alexcohen@gmail.com (Lifetime Email)

On Wed, May 10, 2017 at 9:18 AM, Franz Liem notifications@github.com wrote:

-

re misalignment: this was partially due to a bug in the code. However after fixing this, the contours still don't align perfectly. I have reverted to ROI overlays for now. [image: subcort_contours1] https://cloud.githubusercontent.com/assets/5772811/25900037/57f2de4e-3592-11e7-832f-6d667897e7de.png

new example reports are here https://github.com/fliem/freesurfer/blob/qc_nb/example_reports/00_qc

Due to a nilearn issue https://github.com/nilearn/nilearn/issues/1450 with .add_contours, the white/pial-surface-plots show smaller brains than the subcortical overlays.

Making it an own module or part of niworkflows sounds good to me. For the time being, I have created a command line tool as part of my freesurfer repo, and a docker container. If anyone wants to try it:

docker run --rm -ti \ -v /data/sourcedata:/data/in \ -v /data/freesurfer:/data/out \ --entrypoint=/code/run_qc.py \ fliem/freesurfer:qc_nb \ /data/in /data/out participant

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BIDS-Apps/freesurfer/issues/9#issuecomment-300478829, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPa_4DZV1eRfZxxLEWJNSxnt-8l3o5vks5r4bkUgaJpZM4LUmfj .

satra commented 7 years ago

@akeshavan - perhaps this can somehow be integrated with mindcontrol.

effigies commented 7 years ago

To be clear, the parcellation images are missing for everyone else, right? I'm seeing surfaces and segmentation only.

alexlicohen commented 7 years ago

Yes, I'm not seeing the cortical parcellations either.

-Alex

Alexander Li Cohen, M.D., Ph.D. E-mail: Alexander.Cohen2@childrens.harvard.edu (Medical/Science Email) E-mail: alexcohen@gmail.com (Lifetime Email)

On Wed, May 10, 2017 at 11:30 AM, Chris Markiewicz <notifications@github.com

wrote:

To be clear, the parcellation images are missing for everyone else, right? I'm seeing surfaces and segmentation only.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BIDS-Apps/freesurfer/issues/9#issuecomment-300520102, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPa_59EdmaoBYVrJuaJs7K7B4FzKpyJks5r4dgmgaJpZM4LUmfj .

fliem commented 7 years ago

Sorry, I can't figure out why the parcellations won't show. Here's one of them: sub-01_parc_lh_lateral

PeerHerholz commented 3 years ago

Ahoi hoi gang,

should we revive this given the recent developments over at niworkflows ?

satra commented 3 years ago

this is an entire segment of our current nobrainer collaboration with the freesurfer folks to develop autoqc methods in general and with the short term goal for freesurfer specifically. initial goal discard samples that will not work "well" with freesurfer, medium term goal - modify the samples to run through freesurfer - long term goal make freesurfer faster and more robust to inputs.

this doesn't mean anybody should stop trying to improve manual quality verification. this will also be used in our distributed training mode, where freesurfer runs locally and the lab can help provide feedback as to the quality .

also kwyk can be used in the short term to generate a quality assessment metric for T1 scans in 30s on a GPU.

btw, if anyone is interested in joining the autoqc effort just drop me a line and i will keep you informed about when we meet next (likely in jan, i'm mostly offline over the next few weeks).

PeerHerholz commented 3 years ago

Thx for the update and pointers @satra! I'm all for automatization and happy to not do QC manually (other than to train and inform a model), hehe.

Re the things in this thread: as far as I understood it, the idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right? If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

satra commented 3 years ago

he idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right?

yes, and any useful thing created for the report itself could be used to improve autoqc.

If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

yes. the autoqc stuff can benefit from the report and in turn benefit the report in the future.

Shotgunosine commented 1 year ago

This is an old issue, but I think outputting QA reports would still be a big improvement.

Remi-Gau commented 1 year ago

note that for QA based on freesurfer output there is this: https://github.com/Qoala-T/QC