[ENH] Add electrode coregistration GUI for aligned CT/MRIs for ieeg processing

[alexrockhill]

@larsoner said to open an issue to discuss this, since a new GUI is pretty major maintenance but @adam2392 @jasmainak and others (me) have expressed interest in incorporating this where it would be high quality and have a lot of users as a part of mne-python. I have a manual version written using PyQt5 but it's too slow and Eric suggested a VTK embedding would be much better. This is based on the img_pipe code (cc @libertyh) but has been rewritten and Liberty and Eddy Chang expressed that they would be interested in migrating it into a mature community.

Basically, the GUI would either have users input file paths or navigate using buttons to an MRI, CT (coregistered) and ieeg data file (and optionally a pial surface from a freesurfer recon which helps with ECoG) and then navigate in 3D space to each contact on an electrode/grid and select the channel that corresponds to that location. I'm currently working on a local_max algorithm that almost works that could help make it less work.

I think the main question is whether this would be a good fit in mne-python? cc @agramfort

[adam2392]

Adding some more features that would make it even more highly attractive for iEEG:

• automatic sEEG interpolation backend if at least 2 contacts are manually localized per sEEG electrode (a conference writeup about this procedure here). We simply would have to refactor/test the code.
• we can add a magnet feature; every click near an electrode contact goes automatically to the highest value in an epsilon-ball. fieldtrip localization GUI also has. Easy to implement I presume.
• automatic saving to BIDS :p

[adam2392]

@alexrockhill just spoke w/ @agramfort and @hoechenberger and I think there potentially is interest assuming it doesn't rely on mayvi :p

Agram suggested to us: "let’s get some screenshots/videos for iEEG localization using the GUI to make things more explicit"

cc: @jasmainak

FYI: iEEG Analysis Workflow Generically Looks Like

1. pipeline: handle different imaging, and clinical metadata to get your T1w and CT images lined up in the space you want them.
2. contact-localization (what is proposed here): can be semi-automated depending on EcoG vs sEEG, but always requires a manual GUI step.
3. merging anatomy and iEEG representation: representing the relevant anatomy for each iEEG channel (very important for sEEG case), which can be like temporal-pole, or white-matter-hippocampus, or csf, or ventricle, or skull, which are all mapped from a specific atlas (e.g. the Desikan image from FreeSurfer aparc+aseg.mgz). Note that this step depends on which coordinate system your iEEG locations are in. One could do subject-specific analysis, or map to fsaverage for group analysis, or do some other fancy procedures.
4. Edit: Lastly sEEG visualization: Imo this is super super hard because of the depth problem and what defines a useful viz here. EcoG, easily we can do things like project onto the surface. We have some ideas of porting data into JS to do web-based visualization depending on mne/BIDS data structures.

[alexrockhill]

https://user-images.githubusercontent.com/13473576/111798835-b004d000-8887-11eb-9821-cd75e2b07a2b.mp4

This is the general look

[jasmainak]

hey @alexrockhill and @adam2392 ! Great idea to start a issue. I can't see the video though :( File seems corrupt ...

Do you have an idea how many lines of code a simple GUI-based thing would involve? How big/mature (=how many contributors/users) is the community around img_pipe/seek? I'm asking because I'm trying to figure out which of the following options makes most sense: a) help improve img_pip or seek b) start a new repo (mne-ieeg ?) for this kind of things, or c) port everything to MNE.

Each has its advantages and disadvantages. MNE will give you visibility but each pull request will require careful vetting to ensure nothing else breaks. img_pipe already has an established community (does it?) that can be leveraged but you'll have to probably spend some effort in cleaning the existing code up and building a contributor community (easier for MNE). Starting a new repo has the advantage that everything goes faster, you can control the code quality from the start and things can be tailormade for ieeg/seeg without being bogged down by MNE API. But disadvantage is that community needs to be built and depending on what 3D viz you want to do, you might have to use a bunch of private functions from MNE (not sure though ...)

[adam2392]

In terms of workflow for research and clinical usage related to your question "should X go in MNE-Python":

1. pipeline stuff: imo at least, this doesn't need to be in MNE-Python, or MNE even, but if it was an MNE product, it would definitely gain visibility :p. I think this can either be in the form of nipye in img_pipe, or snakemake in seek-pipeline (oh god plz no custom shell scripts :p). Most pipelines will require FreeSurfer, FSL, SPM12, acpcdetect, and MRTrix3 commands and thus should be ideally containerized in a pipeliney software framework (nipy, snakemake, airflow, prefect, etc.).
2. channel anatomical representation: I think this should be considered for MNE-Python for the reasons we talked about for sEEG. I.e. need to be aware of sEEG general anatomy and which signals might be wm, or ventricle, etc.
3. visualizations: I think certain viz can be in MNE-Python, like the plot_seeg example that Eric helped build out, which can be built on. I feel that JS has significant advantages for more general sEEG 3D viz for developing visualizations for clinical usage and publication figures/videos.

With that said, I think the localization GUI could be a stand-alone MNE.tools app (not in mne-python), with something like mne-ieeg-localization or something? It has mne-python as a dependency to interface with DigMontage for the channel names, ch positions, fiducials and coord_frames. There is strong interest from seek (n = 2 labs :p), since I find going thru fieldtrip to just do this one step cumbersome rather then just having a MNE-interfacing tool. For transparency, I use fieldtrip because we have had a recurring (but rare) issue where there is a left/right hemisphere flip when coregistering Ct -> T1w in other softwares; it's mitigated in Fieldtrip by annotating the LPA, RPA, Zpoint, Nas on the CT based on the electrode implantation.

@libertyh and @alexrockhill can prolly speak more from img_pipe side, but looking at GH, they have labs that consistently use it. Not sure for what portion of it tho (pipeline, localization, or viz), but I'm sure there's more ppl facing this consistent issue of trying to stick w/ Python to run these flows.

[alexrockhill]

Huh the video works just fine on my browser...

Seems like around 30 people have posted issues to/starred/forked img_pipe so I would guess there is maybe a few times that that have been at least interested.

I would be in favor of having a separate package that has everything you need hosted somewhere besides mne that allows users to have more functions to do everything you might need (a lot of that includes wrapping freesurfer commands). But, I think having the core GUI would be really helpful to the broader base of users and it would only be like ~1000 lines, hopefully less.

[adam2392]

@alexrockhill do you mind linking to the repo that houses the GUI you showed us during sprint? I will try to find some time to PR ;)

Might try to get some undergrads too to help out if possible.

[alexrockhill]

Sure it's here https://github.com/alexrockhill/img_pipe, thanks so much for being interested in this. It would be great to have the picker use PyQt for the widgets (which it already does) with a VTK embedded window for the graphics (backend currently matplotlib) as @larsoner suggested if you're looking to do a PR or two.

Eventually maybe we could have the GUI in GitHub and then the rest of the pipeline for convenience either where it is on the ChangLabUCSF github or an mne-ieeg-type package that could be hosted in mne-tools.

[adam2392]

It looks like the main branch hasn't had any updates for awhile, and you have two branches named update and updates :p. Which branch should I reference off of?

@agramfort is it okay for us to stick it into mne-tools now for dev, or wait till later when it's done?

[alexrockhill]

Haha poor naming and I haven't had a chance to work on this for a couple months... update is stable for the manual picker and updates is where I'm trying to add some automation to the picking.

[agramfort]

@agramfort is it okay for us to stick it into mne-tools now for dev, or wait till later when it's done?

can you be more specific? you want to move a repo to mne-tools org? if so sure

[alexrockhill]

Hmmm we could think of a name that maybe pays homage to img_pipe but also should be consistent with mne style...

mne-ieeg is okay but maybe not specific, mne-ieeg-pipeline might be good

Maybe just mne-ieeg is best for brevity

[adam2392]

Perhaps mne-iEEG-localize for the GUI?

[adam2392]

@agramfort is it okay for us to stick it into mne-tools now for dev, or wait till later when it's done? can you be more specific? you want to move a repo to mne-tools org? if so sure

Yep to have an easier time centralizing the eyes of MNE-devs on this push! What is the best way of doing so?

We can tidy up the code from @alexrockhill 's branch and rename it mne-ilocalize? or something that screams iEEG electrode localization :P . In README, we obviously pay props to the original creators in img_pipe, and summarize our new direction (w/ vtk, etc.)

Then we can upload as a new repository in alpha version?

[alexrockhill]

Sounds good to me, just making sure @libertyh is in the loop and this is okay with her also

[larsoner]

Fixed by #9586 I think, feel free to reopen if I'm missing something