ENH: Agenda for fNIRS processing

[rob-luke]

The current status of NIRS support can be seen in this tutorial.

Here is a rough plan for the development of fNIRS processing in MNE.

Refactor existing code:

• [x] Use write wavelengths to info['chs'][ii][loc][9] in addition to the channel name
• [x] Use the info['chs'][ii][loc][9] value rather than channel names in preprocessing

Integrate fNIRS to existing MNE functions:

• [x] Add support for hbo/hbr to plot_topomap
• [x] Add support for hbo/hbr to plot_joint

Channel quality measures:

• [x] Scalp coupling index. (#7215)

Add overlapping channel support to (these may already work with code changes I made, but don't have tests if they aren't ticked):

• [x] topomap (#7414 )
• [x] PSD topomap
• [x] ICA topomap
• [x] TFR topomap
• [ ] Projs topomap

Movement correction

• [x] TDDR with MIT code to work from here (@larsoner and/or @rob-luke)

Interpolation of bad channels:

• [x] Replace with nearest
• [ ] More intelligent interpolation

File support:

• [x] Add SNIRF reader #7972
• [x] Add SNIRF writer
• [ ] Add BIDS support
• [ ] Add NIRSport1 file reader
• [ ] Add NIRSport2 file reader (low priority as device will soon export in SNIRF format)
• [ ] Add Artinis file reader

Implement HRF GLM style analysis Moved to MNE-NIRS till mature enough to merge in Source Analysis Moved to MNE-NIRS till mature enough to merge in

[rob-luke]

Currently I am looking at the temporal whitening tutorial as a starting place for the first step of the GLM implementation.

[larsoner]

@rob-luke WDYT about implementing TDDR? There is MIT licensed code here that we could adapt.

[larsoner]

(Unrelatedly, also added a note about refactoring the wavelength to use info['chs'][ii]['loc'] rather than the channel name)

[agramfort]

if the code becomes too big I would start by putting it in dedicated projects unless there is a consensus so that 80% of people will use this tool anyway

[larsoner]

Agreed, not sure about prevalence. @rob-luke hopefully can comment.

FWIW the code for TDDR is about 20 active lines so not too painful :)

[rob-luke]

@rob-luke WDYT about implementing TDDR? There is MIT licensed code here that we could adapt.

From that style of processing, TDDR is my preferred approach. I would be happy to implement it after the items listed above.

[rob-luke]

if the code becomes too big I would start by putting it in dedicated projects unless there is a consensus so that 80% of people will use this tool anyway

This is handy to know, it will help me think about what to implement in this repo

[rob-luke]

I have put my name next to the items that I have rough code already working for.

[rob-luke]

There is now a python TDDR implementation here.

[agramfort]

MIT license so we can use it in MNE

[rob-luke]

MIT license so we can use it in MNE

Great to hear. Whats the best way to go about adding external code? I assume we don't want to add that repository as a dependency. Do you usually email the author and ask them to add it to MNE, or ask permission for me to add it? Seems like most of the work will be around the data format management which I can do.

[drammock]

@rob-luke We'll want some changes to docstring, code formatting, etc, so to me it would seem odd to ask the original author to do that work for us. But I think it's polite to let them know that you're going to incorporate their work into MNE-Python, and ask if/how they want to be involved and/or credited.

[rob-luke]

Thanks, I'll get in touch with the author and offer to do the bulk of the boring work and ask how they want to be involved/credited.

[rob-luke]

With the 0.20 release coming out soon I thought it would be a good time to say thanks to @larsoner and @agramfort for all the help getting this NIRS integration working. Right now I just completed an analysis ready for publication entirely with MNE. To me it seems all the basic requirements for a NIRS analysis is now present in MNE. Next comes the fun part of developing more advanced analysis/algorithms. Thanks again.

[agramfort]

awesome !

don't forget to give us academic credit :)

https://mne.tools/stable/overview/cite.html

thanks

[rob-luke]

don't forget to give us academic credit :)

Absolutely!!

[larsoner]

Great to hear @rob-luke ! Thanks to you for implementing this stuff, hope to start using it (and contributing more myself) soon :)

[rob-luke]

if the code becomes too big I would start by putting it in dedicated projects unless there is a consensus so that 80% of people will use this tool anyway

I am going to implement a few lesser used algorithms now. @larsoner is mne-project-template still the recommended way to start an extension?

[agramfort]

yes +1 for mne-project-template

[rob-luke]

if the code becomes too big I would start by putting it in dedicated projects unless there is a consensus so that 80% of people will use this tool anyway

Ok I have created MNE-NIRS to fulfil this need.

I have initially added Cui et. al. 2010. And some HRF code. I will use this repo as a testing ground for code and anything that is widely requested can be moved to MNE. I will try and keep the same standards as you enforce in this repo.

Cui, Xu, Signe Bray, and Allan L. Reiss. "Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics." Neuroimage 49.4 (2010): 3039-3046.

[agramfort]

cool !

let us know if you want to host this repo in the https://github.com/mne-tools/ organization

[rob-luke]

Yeah that would be great. That should help people find it, which should increase usage. I will hit transfer ownership in the settings.

[rob-luke]

I tried to transfer the repo but got an error message saying You don’t have the permission to create public repositories on mne-tools. There is no rush on this though, we can just transfer it whenever suits you.

[agramfort]

give me admin access to your repo and I'll do the transfer

[agramfort]

You should now be able to do it yourself. It seems I cannot have admin rights on your personal repo.

[rob-luke]

Thanks @agramfort I accepted the invite. Unfortunately I still get the same error. I will go grab some dinner and hope that permissions propagate. Worst case I transfer ownership to another organisation that I have access to, then give you rights there to transfer it. Sorry about the stuff around.

[agramfort]

here we go https://github.com/mne-tools/mne-nirs

[rob-luke]

Thanks. I’ll fix all the links, ci, etc over coming days.

[larsoner]

A general HRF/deconvolution function already seems like it would be general enough to potentially put in MNE. But it's worth checking to see if some fMRI package in the Python ecosystem provides it already. A quick search suggested a couple of potential starting points:

• rsHRF\
• DIPY

Maybe you can just wrap one of these? If it's all that's required, we already have a soft dependency on DIPY so that would be very easy to add here...

[rob-luke]

@Larsoner I have started thinking about that as a WIP here https://mne.tools/mne-nirs/auto_examples/plot_10_hrf.html

Currently I am just trying to settle on an API. And was wrapping pybids, but will look in to the packages you suggested.

The hrf/deconv approach I would like to have available in MNE is from Barker and Huppert (I linked the paper in checklist up the top). I have been using their ARIRLS approach for several years and find it extremely robust.

https://github.com/huppertt/nirs-toolbox/blob/master/%2Bnirs/%2Bmodules/AR_IRLS.m

https://github.com/huppertt/nirs-toolbox/blob/master/%2Bnirs/%2Bmath/ar_irls.m

[rob-luke]

Ok I took another look around and NIPY looks the way to go. It also has some fMRI code that I foresee being helpful (e.g. http://nipy.org/nipy/api/generated/nipy.modalities.fmri.design.html).

@larsoner what does soft dependency mean?

[agramfort]

for designs I would look at https://nistats.github.io/

nipy stats is not a very active code base and nistats will soon be merged into nilearn.

[rob-luke]

Thats exactly what I needed @agramfort, somehow I had missed nistats. Thanks

[rob-luke]

Ok I got carried away and have made a first pass at the HRF style analysis based on nistats at...

https://mne.tools/mne-nirs/auto_examples/plot_10_hrf.html

And created a meta issue of this meta issue to track my thoughts. I pinged you both as an FYI.

[larsoner]

Awesome example! I've subscribed to the repo :)

[rob-luke]

Thanks. I will continue the HRF analysis development in MNE-NIRS and once matured we will merge in to MNE.

[kylemath]

Glad I found this great discussion, I have an imagent from ISS and am working on a function to load its phase and intensity data into a MNE format similar to the NIRx.py loading example. We are thinking of moving our code base for fast optical imaging from matlab into python with these tools (currently at github.com/kylemath/mopt3d). Look like you all have thought about many of the major hurdles already, I think I gathered I should work from this new repo: https://github.com/mne-tools/mne-nirs?

[larsoner]

@kylemath the plan at this point is to support all of the stuff at the top of this issue directly in MNE-Python except what is marked as being for mne-nirs. Mostly that means I/O of fNIRS data is in scope for MNE-Python, but HRF deconvolution is being worked out right now in mne-nirs. Basically mne-nirs is there to allow @rob-luke and other to try things out and figure out APIs, and once they are settled, things should migrate to MNE-Python.

I think I gathered I should work from this new repo: https://github.com/mne-tools/mne-nirs?

I would say you should try implementing your mne.io.read_raw_imagent or mne.io.read_raw_iss here first, probably with guidance from @rob-luke. Once that's it, talk with him about what fits better here or in mne-nirs based on what is standard (enough) practice in the field.

At least that's how I see it! Feel free to clarify @rob-luke @agramfort

[rob-luke]

Hi @kylemath, I agree with @larsoner, lets get the imagent reader straight in to MNE. I would recommend including at least the following files:

• Data reader: nirx example
• Test data with documentation about how it was created: nirx example 1, example 2
• Reader tests: nirx example

MOpt3d looks great. I think that kind of functionality is more suited to MNE-NIRS than MNE for now. Particularly photon migration is something I would love to get up running (see issue). A quick stalk of your repository reveals that you have extensive experience in this style of analysis. I am trying to sketch out a roadmap over at MNE-NIRS by opening issues, please feel free to create/comment on issues to share your experience of what works or doesn't, or what functionality you think is essential or not.

[kylemath]

@rob-luke @larsoner - My student and I are working on the function to load BOXY data files from the imagent software, along with digitized sensor location files and lists of location names for our helmets. This is working quite well at https://github.com/kylemath/pyoptical. We compute the transformation matrix needed to go from our sensor space into the fsaverage space, and add it to the info object to get added to the Raw data objects.

ch_pos, ch_names = ourLoadFunction(datafile)
montage = make_dig_montage(ch_pos, fiducials)
info = create_info(ch_names, srate, 'eeg').set_montage(montage)
mri_fiducials = read_fiducials(fsaverage)
trans = coregister_fiducials(info, fiducials)

We noticed that the nirx function does things a bit different, by precomputing a transformation matrix from fsaverage, applying it to the fNIRS sensor coordinates and outputing those TRANSFORMED VALUES without a transformation matrix. Why does NIRx.py apply the transform ahead of time? Instead of leaving it up to the user?

[larsoner]

Why does NIRx.py apply the transform ahead of time? Instead of leaving it up to the user?

Channel locations in MNE-Python are supposed to be stored in Neuromag head coordinates, see:

https://mne.tools/dev/auto_tutorials/source-modeling/plot_source_alignment.html https://mne.tools/dev/overview/implementation.html#the-forward-solution

So the read_raw_nirx transforms them to the head coordinate frame. Since the NIRx are in MNI (apparently?) and fsaverage is in MNI, you can just use fsaverage's standard transformation to get them to (fsaverage) head coordinates.

[larsoner]

... but see also the evolving discussion here https://github.com/mne-tools/mne-python/issues/7691#issuecomment-621424988

[kylemath]

Ok thanks, I think part of our confusing is that we only have two coordinate frames, and our "head" and "MEG" frames are aligned.

1) We digitize 300 points on the optical imaging headgear with a subset of those being channel locations (sources or detectors), as well as around the eyes, and fiducials. I am assuming this is our "head" frame.

2) A subset of these points are also the locations of light sources and detectors, which I assume is like the "MEG" frame, but is already co-registered with our "HEAD" frame

3) Then we have an average or subject specific MRI parcellation of the surface of the head, "MRI" frame

So I guess this is more analogous to the EEG coregistration in that regard. Is there some discussion of this EEG coregistration case somewhere I can't see, I see this, but not how to get the trans matrix: https://mne.tools/stable/auto_examples/visualization/plot_eeg_on_scalp.html

[rob-luke]

Hi @kylemath,

I will leave these coordinate questions to the others. But it might be easier if you open a pull request as a WIP in this repository for the imagent data reader, that way we can comment directly on the code and iterate faster.

[kylemath]

yes good idea, will try to organize that this weekend after some testing

On Thu, Apr 30, 2020 at 4:43 PM Robert Luke notifications@github.com wrote:

Hi @kylemath https://github.com/kylemath,

I will leave these coordinate questions to the others. But it might be easier if you open a pull request as a WIP in this repository for the imagent data reader, that way we can comment directly on the code and iterate faster.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mne-tools/mne-python/issues/7057#issuecomment-622177322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA36GFPFYH5AVRHDJH4U43TRPIEK5ANCNFSM4JNDSW5Q .

-- Kyle E. Mathewson, Ph.D. Assistant Professor - Department of Psychology, Faculty of Science Director - Attention Perception and Performance Lab Affiliate - Neuroscience and Mental Health Institute, Faculty of Medicine and Dentistry University of Alberta P455 - Biological Sciences Building 11455 Saskatchewan Dr. Edmonton, Alberta, Canada, T6G 2E9 Phone: 1-780-492-2662 Email: kyle.mathewson@ualberta.ca Web: www.kylemathewson.com

[rob-luke]

Great, didn't mean to rush you, just whenever you have time. I found the support from the MNE team very encouraging when I added the NIRX reader.

Opening a PR will also move the detailed conversation there and keep this issue more for higher level agenda discussion.

[kylemath]

Moved this discussion to: https://github.com/mne-tools/mne-python/pull/7717 @rob-luke

[larsoner]

@rob-luke this is marked for a 0.21 milestone. Could you look into the remaining plotting issues above in the next couple of weeks so that we can tick a few more boxes for 0.21?

[rob-luke]

Sure I can take a crack at PSD and TFR, but I don't know what projs is and if its relevant to NIRS. The other file type readers are also out of my control, particularly the artinis won't be done by 0.21. Do you plan to create another meta issue after 0.21 for the outstanding issues? I think we could ditch the mega meta style list as everything is running pretty well now and just tackle issues as they arise.

[larsoner]

Projs we can tackle at some point if people want to try using it as a means of noise suppression. Agreed we can just open PRs for readers as necessary