search

brainstorm-tools / brainstorm3

Brainstorm software: MEG, EEG, fNIRS, ECoG, sEEG and electrophysiology
http://neuroimage.usc.edu/brainstorm
GNU General Public License v3.0
385 stars 162 forks source link

Automatic EEG localization and labelling using Revopoint 3D #716

Closed chinmaychinara91 closed 2 weeks ago

chinmaychinara91 commented 5 months ago

Tutorial: https://neuroimage.usc.edu/brainstorm/Tutorials/TutDigitize3dScanner#Digitize_using_Revopoint_Digitize_GUI_in_Brainstorm

Test Data: Can be downloaded from here.

Goal is to allow automatic localization and labelling of EEG caps using 3D scanned mesh of a person wearing the cap (captured using Revopoint 3D). This represents a new low-cost, fast, easy-to-use approach for digitizing heads and electrodes as compared to the traditional digitizer.

NOTE: Currently tested well with two configurations of EEG caps: https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/d40c719ede6ee167c290c7b526836d57811761ca https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/c49ebc6cc474979bfd634731f5be67dcca33e6bc

Proposed GUI within Brainstorm LEGACY MODE Screenshot 2024-09-27 113338

2024 MODE Screenshot 2024-09-27 113125

@tmedani @ajoshiusc @yashvakilna

chinmaychinara91 commented 4 months ago

tutorials in GUI and tutorial scripts

Can you point me to this ?

rcassani commented 4 months ago

Can you point me to this ?

@chinmaychinara91

1. Running tutorials on GUI

There are many tutorials in this page, that you can follow to verify the changes are backwards compatible https://neuroimage.usc.edu/brainstorm/Tutorials/

2. Running tutorials with scripts

Most tutorials can be run with scripts, located in main Brainstorm repository. Look for the tutorial_*.m scripts: https://github.com/brainstorm-tools/brainstorm3/tree/master/toolbox/script

There is also this private repo: https://github.com/brainstorm-tools/bst-tests

rcassani commented 4 months ago

Hey guys, unfortunately, we do not have a revopoint yet. Can you provide the following to tests the code in the PR?

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

chinmaychinara91 commented 4 months ago

Hey guys, unfortunately, we do not have a revopoint yet. Can you provide the following to tests the code in the PR?

  • Some raw files to test the import of the new surface format
  • A protocol where the sensor localization has been already performed

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

@rcassani:

  1. Based on the tutorial in the description above, raw test data is attached here as sub1_Waveguard_ANT_65.zip
  2. The test protocol can also be found in the link above as TutorialDigitizer.zip

I ran it in the simulated mode without the Polhemus connected. Functionalities looked good and nothing broke. Polhemus here at USC has gone for some repair. Once it is back I will check it.

Also ran the exhaustive testing with various tutorials as you instructed above. All looked good.

tmedani commented 4 months ago

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

we can test that at USC, we have the digitizer around.

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

rcassani commented 4 months ago

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

chinmaychinara91 commented 4 months ago

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

Yes definitely. We will test it here and let you know.

tmedani commented 4 months ago

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

Yes definitely. We will test it here and let you know.

It may require some time to do that on the digitizer itself. we need to set it up and run it.

rcassani commented 4 months ago

@Moo-Marc, can you give us a hand with the test with the Polhemus?

chinmaychinara91 commented 4 months ago

It may require some time to do that on the digitizer itself. we need to set it up and run it.

Will be testing the PR with Polhemus today. Will keep posted.

chinmaychinara91 commented 4 months ago

The Polhemus here at USC is being bit of a task as it just got back from repairs. @Moo-Marc could you help us in testing this PR out at your end ? Just want to make sure already available functionalities do not break.

Moo-Marc commented 3 months ago

Hi, not sure why but I had not received notifications for your messages. Sure I'll help test. Hopefully we don't get conflicts with my other digitizer PR.

chinmaychinara91 commented 3 months ago

@Moo-Marc thank you so much. Do let me know once you test it out.

chinmaychinara91 commented 3 months ago

d753abd commit takes into account that the New Digitize Panel changes (https://github.com/brainstorm-tools/brainstorm3/pull/682) are merged to this PR.

chinmaychinara91 commented 3 months ago

@rcassani this is ready for review. Both the legacy and new panel support Revopoint now. Do test it out on your end based on the tutorial in description and let me know if you face any issues.

chinmaychinara91 commented 2 months ago

@rcassani thank you for your comments. I have replied to them individually. Do go through them. But to summarize yes it needs another round of cleanup from my end which I will work on this week. Definitely it is not generalized well and there are some hard coded values but we are yet to receive more samples with variety of caps to get a good sense of generalizing them.

rcassani commented 2 months ago

@chinmaychinara91, related to 53f43dd.

Do not patch code just to make it work. You have to understand why there was a problem, and how it can be solved from origin.

Moreover, remember to keep previous behaviours otherwise:

In addition, it would need to update the usage strings for panel_digitize and panel_digitize_2024

What about solving the bug from root like below? This solution is compatible with former ways using 'Start' with panel_digitize and panel_digitize_2024

--- a/toolbox/sensors/panel_digitize.m
+++ b/toolbox/sensors/panel_digitize.m
@@ -40,7 +40,7 @@ end
 %#function icinterface

 %% ===== START =====
-function Start(DigitizerType) %#ok<DEFNU>
+function Start(varargin) %#ok<DEFNU>
     global Digitize    
     % ===== INITIALIZE CONNECTION =====
     % Intialize global variable
@@ -68,10 +68,11 @@ function Start(DigitizerType) %#ok<DEFNU>
             'trans',    []));

     % ===== PARSE INPUT =====
-    Digitize.Type = 'Digitize';
-    if nargin > 0 && ~isempty(DigitizerType)
-        Digitize.Type = DigitizerType;
+    DigitizerType = 'Digitize';
+    if nargin > 0 && ~isempty(varargin{1})
+        DigitizerType = varargin{1};
     end
+    Digitize.Type = DigitizerType;
     switch DigitizerType
         case 'Digitize'
             % Do nothing

Same thing must be done for panel_digitize_2024

chinmaychinara91 commented 2 months ago

@rcassani

I have made all the changes based on the comments and discussions above and on Slack. Let me know if you find any errors on your side.

To summarize:

  1. User can now just right click on the imported texture surface > Digitize (3D Scanner) to start the digitization.
  2. Regarding There is not a clear workflow for the case where the subject has its own anatomy, the user can right click on the subject > Import surfaces and import the textured surface. Then right click on the imported textured surface > Digitize (3D Scanner) to start the digitization. follow step 1 above.
  3. User now gets to choose what to do when starting 3D Scanner digitization (Use existing surface or Add a new surface).
  4. Better way of sampling without having to use reducepath.
  5. auto_3dscanner now handles all the automatic detection and labelling of 3D Scanner acquired mesh. For new caps, we just need to change the getEegCapLandmarkLabels function and that should be it.
  6. Did some rigorous testing and was able to come up with a single range for imfindcircles with the caps we have available. Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/3403c9ef4359220ca9b189425b14ea901ef3f079.
  7. Updated the cap templates for Colin27 and ICBM152. Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/c49ebc6cc474979bfd634731f5be67dcca33e6bc.
  8. The textured surface in database is updated with Digitize.Transf transformation once the fiducials are digitized.
  9. Had forgot about this https://forum.revopoint3d.com/t/revopoint-generated-mesh-coordinate-system/25662. The Revopoint vertices are in 'mm'. Did some digging here https://www.fieldtriptoolbox.org/tutorial/electrode/ and I think we need this approach https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/b1124bf6e0fc558fd81ffb303ff36d0c4210859d if we want to support any generic head surface acquired through other devices. Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/06df2526657c09def91b5418ff32d3b4c9e9d958 https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/cedc8affd4852a161db8b3a486b3524d18043024 (fieldtrip dependency removed).
  10. Needed to do some ordering changes in the way the initialization points were captured for scalability. Updated the tutorial with the instructions.
  11. For https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/86e8d830c69c514b82b2569632854d9797543757, the data to test with can be found in the Test data link in description.
    • ft_obj: test data provided by fieldtrip here
    • 3d_software_obj: random freely available head scans online

Tutorial updated partially. Should be good for your testing. Will do a final update of tutorial after it is ready to merge.

chinmaychinara91 commented 2 months ago

For importing an OBJ into brainstorm:

  1. The in_tess_wftobj function in actually a slightly modified submodule of https://www.fieldtriptoolbox.org/reference/fileio/ft_read_headshape. Should we use the fieldtrip function directly or since the license allows us, just continue to do it as it is right now (and just put a reference to the fieldtrip function).
  2. We also use the https://www.fieldtriptoolbox.org/reference/forward/ft_convert_units function to determine the units of the vertices based on the shape of the head. This is a good function to have.

Both these functions are available as part of spm12 plugin (/external/fieldtrip) and also the fieldtrip plugin. How do we go about this ?

@tmedani

rcassani commented 2 months ago

In short, it does not makes too much sense to request the user to install FieldTrip as plugin and Load it to perform trivial operations and things that already in the Brainstorm source code. Moreover, not all functions in FieldTrip are available in the compiled version of Brainstorm.

  1. The in_tess_wftobj function in actually a slightly modified submodule of https://www.fieldtriptoolbox.org/reference/fileio/ft_read_headshape. Should we use the fieldtrip function directly or since the license allows us, just continue to do it as it is right now (and just put a reference to the fieldtrip function).
  1. We also use the https://www.fieldtriptoolbox.org/reference/forward/ft_convert_units function to determine the units of the vertices based on the shape of the head. This is a good function to have.
chinmaychinara91 commented 2 months ago

@rcassani thanks for pointing out the resources. That makes things much simpler. I don't think we will need FT now.

chinmaychinara91 commented 4 weeks ago

Code looks good, almost ready to merge.

  • Most of the code duplication mentioned in the previous review has been addressed.
  • @chinmaychinara91 I have addressed several of the comments in this review. Please take a look to those changes so you can improve future PRs

@rcassani thank you for the review. I will work on the comments you gave and get back.

rcassani commented 3 weeks ago

@chinmaychinara91, @tmedani It seems this PR is ready.

Please add a warning in the tutorials and in the GUI (e.g. when clicking on the Auto button) that the Auto feature is experimental, and results must be carefully verified.

chinmaychinara91 commented 3 weeks ago

Please add a warning in the tutorials and in the GUI (e.g. when clicking on the Auto button) that the Auto feature is experimental, and results must be carefully verified.

Agreed will do

chinmaychinara91 commented 3 weeks ago

Did the tests. All look good now. It can be merged.

chinmaychinara91 commented 3 weeks ago

Tutorials updated. Found some more bugs while doing some tests and fixed them in https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/53f04ca38eab3370a2b707eef5485ed7af60cd44 and https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/bcacbcaea9a71017d82ecb3f112bb3f89c1a59b4. It is good to go now.