Miccai/Oasis Label Set Progress and Questions

[butellyn]

I have created a branch with my work on getting the Miccai/Oasis label set into ggseg: https://github.com/butellyn/ggsegExtra/tree/master/data-raw/MiccaiOasis

MICCAI 2012 Labeling Challenge: http://www.neuromorphometrics.com/2012_MICCAI_Challenge_Data.html

The nifti of these labels was made by using joint label fusion to warp the labels from the OASIS brains to MNI 152 space.

I have made a lot of progress on getting these labels into ggseg, but I have run into a few issues along the way that may turn out to be big problems. I have never used freesurfer or connectome workbench before, so it's been quite a learning curve. I'm sorry for the issues that probably stem from my misunderstanding of freesurfer and connectome workbench. I have tried to answer my questions through exploring various listservs and stackoverflow.

I know you are working on an R function to get atlases into ggseg, which is awesome. If you don't have time to answer my questions below, or try running my code, could you try putting this atlas in using your new function? I am trying to make a figure for a paper that got an R&R. The fact that I can use ggplot to organize the figure is great for me, and for the person for whom I am making the figures. (I am trying to put together a very complicated figure that I need to be able to edit frequently and quickly.)

The main thing I am stuck on is mkMic.R (analogous to mkHo.R). One of the dependencies for one of the libraries loaded in mkHo.R is V8. Unfortunately, it failed to install and returned the following error message: trying URL 'https://cran.mtu.edu/src/contrib/V8_3.0.1.tar.gz' Error in download.file(url, destfile, method, mode = "wb", ...) : cannot open URL 'https://cran.mtu.edu/src/contrib/V8_3.0.1.tar.gz' In addition: Warning message: In download.file(url, destfile, method, mode = "wb", ...) : cannot open URL 'https://cran.mtu.edu/src/contrib/V8_3.0.1.tar.gz': HTTP status was '404 Not Found' Warning in download.packages(pkgs, destdir = tmpd, available = available, : download of package ‘V8’ failed

In addition to this error, mri_vol2surf does not seem to exist in any of the libraries loaded in mkHo.R. I installed and loaded the freesurfer package for R, but this does not have mri_vol2surf either (but it does have mri_surf2surf). It doesn't look like you use test.*r.mgh later in the script, though, so I am not sure what the purpose of this line is. It also seems as if it is doing the exactly same thing as line 16 in miccaiOasisGGSEG.sh.

---

Below I detail the problems I ran into while adapting your scripts for the Harvard-Oxford cortical atlas to the Miccai/Oasis label set. I have annotated my scripts some, but I think the full thought process may be useful for debugging, or for other users who are interested in getting an atlas into ggseg. (Note that I do not use relative paths throughout the script, aside from in R. OUTDIR, INDIR and SCRDIR were originally paths on my computer, but I changed them to pwd for portability. They should probably become command-line arguments.)

The first major issue I ran into had to do with indexing. Freesurfer seems to expect the background to be coded as 0, and all other indices to be consecutive integers. My atlas was not coded this way. To get it closer, I subtracted 99 from all cortical ROI indices because the indices were 100-207. Unfortunately, a few of the indices were intentionally skipped because labels that were originally planned to be part of the atlas were not included in the final version. This made the whole process of re-indexing to consecutive integers a daunting/error-prone task. When I loaded these labels into freeview, I originally created the *h.mic.annot files using the following commands: sudo mris_label2annot --sd ${SUBJECTS_DIR} --s fsaverage --ctab ${OUTDIR}/miccaiCtab.txt ${LABS_L} --h lh --a mic sudo mris_label2annot --sd ${SUBJECTS_DIR} --s fsaverage --ctab ${OUTDIR}/miccaiCtab.txt ${LABS_R} --h rh --a mic miccaiCtab.txt contains indices for both left and right hemispheres, and the indices match the indices in ${INDIR}/mniMICCAI_LabelsCortical0.nii.gz. Unfortunately, freesurfer clearly was not reading the indices from the idx column (which I created to match freesurfer's color lookup table; I tried a color look up table like the one created for Harvard-Oxford, but that failed with some error that I do not remember). I know this because it only read the number of lines in miccaiCtab.txt that corresponded to the number of ROIs in a given hemisphere. Then I split miccaiCtab.txt into miccaiCtab_R.txt and miccaiCtab_L.txt for left and right. Errors persisted (I couldn't open the labels in freeview), and I guessed that it may have to do with the fact that the indices were no longer (largely) consecutive integers. So I changed the indices in miccaiCtab_R.txt and miccaiCtab_L.txt, and it mostly worked. I could now see my labels in freeview. But the labels did not line up with the correct regions. At this point, I decided that this probably should not matter, since I could redefine index-label pairings in the final tibble for ggseg. I hope I was correct about this!

Then I ran into an issue at line 66 in miccaiOasisGGSEG.sh. The outputs of mris_convert did not match the inputs to smooth_labels.sh in the instructions, so I had to change this. Hopefully I interpreted the intention here correctly. In smooth_labels.sh, I had to change to the key flag for -gifti-label-to-roi. I also couldn't get gnuparallel working on my computer, so I replaced those lines with code that produced what I think is the same output. Of concern, when I finally got to mkPics.sh, I noticed that my ROIs were still very choppy, so I think smooth_labels.sh probably did not work. I didn't run into any errors with my final version of smooth_labels.sh, but some parameter probably needs to be adjusted to make the labels even more smooth.

Other little problems showed up throughout, but these are the major ones. Please let me know if I can be of help with any documentation, or if you would like me to clarify anything about what I have done.

[butellyn]

Some of the ROIs on the medial wall are creeping onto the opposite hemisphere, too. Have to figure out how to reassign those voxels.

[butellyn]

Was able to install V8 with help from the package creators.

[butellyn]

The main problem now is that my labels still have many holes. I tried upping the wb_command -metric-dilate to 20 (from 2), but that hardly improved the situation. I also removed wb_command -metric-remove-islands, but that didn't help much. Here is what it looks like right now.

Any advice you have to improve this is greatly appreciated!

[drmowinckels]

Hi @butellyn ! Thanks for being so thorough and keeping us posted on your progress. I have a conference next week I really need to prep for, which is why I have been so silten. I'm sorry about that. But I expect after next week I will have more time and can look into this with you more closely.

If it helps, this is as far as I also got with this procedure. My maps looked a little less holey, but all-in-all much the same. I have an idea though of how to improve it, though.

Unfortunately, an untimely necessary formating of my hard-drive made me loose the work I had done on getting the functions in R working (stupid me for not pushing to github). I am pretty close to recreating the work, and the functions are almost working again. After the conference, I will work more on them, and share with you my progress and we can have a discussion on how we can improve your atlas.

[butellyn]

Oh no! Looking forward to brainstorming more!

A PI in my lab suggested dilating the labels, which helped a lot: fslmaths ${INDIR}/mniMICCAI_LabelsCortical0.nii.gz -kernel sphere 3.5 -dilD ${INDIR}/mniMICCAI_LabelsCortical0_dilated.nii.gz

Now the labels that are on the medial wall that were bleeding over to the opposite hemisphere are more apparent, though. Do you know how to combine two labels?

Aesthetically, it would be nice to smooth out the edges so that they aren't so jagged.

And then at least two labels are mysteriously missing (lh_LiG, rh_IoG). Looking into that now.

[butellyn]

So it turns out the missing labels has something to do with the kernel. When I changed it to 2, no regions where missing on the left hemisphere, but there are more gaps than desirable. The right hemisphere still had at least one missing label: fslmaths ${INDIR}/mniMICCAI_LabelsCortical0.nii.gz -kernel sphere 2 -dilD ${INDIR}/mniMICCAI_LabelsCortical0_dilated.nii.gz

I ended up deleting the labels that bled across hemispheres. It makes the edges of the images a little jagged, but it's better than the alternative. Would be nice to figure out how to fix this.

Here's what the (tentative) figure looks like!

[butellyn]

A senior research scientist in my lab suggested using ImageMath from ANTs PropagateLabelsThroughMask to fix the holes. I will try this as soon as I manage to compile ANTs on my local computer.

[drmowinckels]

Hi @butellyn ! Very happy you're keeping us update on your progress. I'm sorry I'm not as active as I'd like in this, but we are both of us very much swamped in other work at the moment. I hope to get a respite soon so I can follow you up on this. I think the tentative figure is looking very promising.

[butellyn]

@Athanasiamo, have you had a chance to look into this? I'd love to close these gaps very soon, if possible. If not, I can catch it on revisions.

[drmowinckels]

Hi @butellyn! I have looked at it, but not yet made any real progress. But I am finding very little time to spend on this at the moment, with other pressing issues. I give it a go every once in a while and make small steps, but not to the extent I'd like to!

[drmowinckels]

Hi @butellyn ! Sorry for the lateness of this, but we have finally made a workflow that seems to work well for cortical parcellations. The workflow is quite different, but is done mainly through R (as long as other software is on your system) and should work well.

We found that working through Freesurfers tcl scripts became too difficult to control, and since macOS catalina no longer supperts 32-bit programs (tcl) is not usable several users (me among them). So we have opted for a workflow that will first help you make a ggseg3d-atlas, then use that atlas to make the ggseg-atlas. This has worked well for our attempts so far, no holes or weird segmentation issues.

We'd love you to have a look and see if that at all is a workflow that works for you.

https://lcbc-uio.github.io/ggsegExtra/articles/customatlas.html

[butellyn]

Awesome! We ended up editing the holes out in Photoshop, but it is totally possible I'll have to remake the figure, in which case I'll definitely use this. I'll try to give it a test run soon. Sorry for the late response. This pandemic is a lot!

[drmowinckels]

if you manage making an annotation file, I'd love the atlas as part of the ggseg-family of atlases. So if you do get it working, please let ut know and we'll help you contribute an atlas :)

[butellyn]

@Athanasiamo And I have to remake the figure! Unfortunately, I had since updated my version of R to 3.6.2, for which ggsegExtra does not seem to be available. Would it be possible to make a version of ggsegExtra available for this version of R? Thank you!

[drmowinckels]

Hi @butellyn ! It should absolutely be available, but I suspect you have tried installing it from cran, where it is not available. You need to do remotes::install_github("LCBC-UiO/ggsegExtra", build_vignettes = FALSE)

[butellyn]

Ah! Whoops! Thank you!

[butellyn]

The atlas creation fails on step 3. Here is the call that I am using: make_aparc_2_3datlas(annot="mic", subject="fsaverage"). This is the error message I am getting: "cannot find specified surface file for this subject. Please check file path for surface: /Applications/freesurfer/subjects/fsaverage/surf/rh.LCBC Error:". It looks like you probably have your own custom surface called rh.LCBC that does not exist in the freesurfer package. It looks like I can just download them from here (https://github.com/LCBC-UiO/ggsegExtra/tree/master/inst/surfaces), but I think it would be helpful to have them install as part of the command.

[butellyn]

Got a little bit further. I think I should be using surface="inflated"

[drmowinckels]

I'm surprised you got a lot further. The LCBC surface would have been created for you (without extra effort) if you had chosen to use 'fsaverage5' as a subject, which is what we recommend. 'fsaverage' is alot of vertices and will create a plot that is very slow to render in the end because it is so large.

[butellyn]

Ah, I had to switch to 'fsaverage' because that is the surface I made my annot files in the last time I took a crack at all of this. Maybe it is time to go back and recreate those.

[butellyn]

When I switch to the inflated surface (make_aparc_2_3datlas(annot="mic", subject="fsaverage", surface="inflated")), it runs to completion without any errors, but then when I use make_ggseg3d_2_ggseg, I get the following error message: "Atlas must be a valid ggseg3d-atlas Check atlas with is_ggseg3d_atlas()Error:". Is there a reason not to use the inflated surface? If so, what is the reason? And in what way may it result in an invalid ggseg3d atlas?

[drmowinckels]

Ah sorry, I see now there is some lack of information in the tutorial to make atlases. Thanks for pointing that out. Because the data usually need some post-processing (removing uneeded structures, renaming of things etc), the final output from the make_aparc_2_3datlas() function is not saved as a ggseg atlas.

Before using the next function, try doing as_ggseg_atlas(mic) , that should in theory do it.

[butellyn]

Thanks! I tried out the as_ggseg_atlas function, but got the following error: There are missing necessary columns in the data.frame for it to be a ggseg_atlas: '.long' '.lat' '.id' 'area' 'side'. Are these supposed to be created with the make_aparc_2_3datlas function, or do I need to create them separately?

[drmowinckels]

there are some gremlins here. I hope to get time to look further into this next week.

[butellyn]

Hey, just checking in on this. Our paper came out, but now everyone wants to keep using the atlas I made, so it would be nice to figure out how to permanently fix the holes.

[drmowinckels]

Hi @butellyn !

We just released the package with new pipelines and functions for creating atlases. If you managed to create an annotation file before, you should easily be able to use a process similar to what we describe here: https://lcbc-uio.github.io/ggsegExtra/articles/createdkt.html

[butellyn]

SO CLOSE. I love these new functions, just running into two problems.

1.) make_aparc_2_3datlas() fixes almost all of my holes, but there are still a few left. There's also a few spots where one label is popping up in the middle of another. Suggestions for how to fix this?

2.) I am running into an error with make_ggseg3d_2_ggseg(). Error: Error in system2("orca", "--version", stdout = TRUE, stderr = TRUE) : error in running command. Do I need to have have a program called "orca" installed?

Thanks!

[drmowinckels]

Great that the new functions are working fairly well!

1) I am a little unsure how to fix this. It looks like there are vertices that are not mappen in your atlas somehow. It's the only reason I can think of the holes popping up like this. Alternatively, if you converted the atlas from another file, perhaps there is an issue with the atlas conversion to annotation file? How does the annotation file look when plotted in freeview?

2) orca, yes! Actually, I forgot that that function calls orca. So I'll need to fix the documentation on that (and also add in a check of orca availability before running the function)

[butellyn]

I think there are vertices that aren't mapping to the surface. These are the same annot files that got turned into the plot I created on January 30 above. One of the major issues I was running into was trying to figure out how to fill the holes. It seems like make_aparc_2_3datlas() filled some of the holes, but not all of them. Does it have the functionality? If so, is there a way to adjust a parameter to make it more aggressive? Here's my fork from forever ago with all of the files: https://github.com/butellyn/ggsegExtra

I tried viewing the surface in Freeview, but I got the following error: "Did not find any volume info ��ؙ�reading colortable from annotation file... colortable with 50 entries read (originally /Users/butellyn/Documents/hiLo/data/ggseg_miccai/output/miccaiCtab_R.txt) colortable with 50 entries read (originally /Users/butellyn/Documents/hiLo/data/ggseg_miccai/output/miccaiCtab_R.txt)"

After running /Applications/freesurfer/7.1.1/bin/freeview -f /Applications/freesurfer/7.1.1/subjects/fsaverage5/surf/rh.pial:annot=mic.annot:name=pial_aparc:visible=0

[drmowinckels]

I completely lot track of this issue, I'm terribly sorry. How far did you get in the end?

[butellyn]

Haha, no problem. I am actually no longer working on the project, because I transitioned from a data analyst role to being a PhD student at Northwestern. One of my replacements, Noah Hillman (@Nchillman, https://github.com/Nchillman), is working on this now.