MouseLand / facemap

Framework for predicting neural activity from mouse orofacial movements tracked using a pose estimation model. Package also includes singular value decomposition (SVD) of behavioral videos.
https://facemap.readthedocs.io
GNU General Public License v3.0
156 stars 61 forks source link

Applicable to inverted images (2P illumination through pupil)? #39

Closed generalciao closed 1 year ago

generalciao commented 2 years ago

Has anyone tried to to use Facemap on face movies where the pupil is bright (due to transcranial 2P laser illumination)? Maybe in combination with an "invert" preprocessing step?

7rebor commented 2 years ago

I did originally try that, but you need to get the timing right so that the frame rate of the camera and the microscope are synced. If they weren't synced the pupil was changing size due to the scanning of the laser being at different points (or maybe laser blanking occuring etc.) causing the contrast of the pupil to change. I did an inversion step before facemap in ImageJ.

Another problem I think that arises is that the mouse moves its eye and the light is refracted in different ways, changing the contrast from frame to frame and therefore changing the estimation of pupil size and position.

We bought a crappy security camera IR light from Amazon for very cheap (something like this https://amzn.to/3GXxTdJ) and it works great. It only comes on when it is dark (microscope box is closed) - would recommend doing that to avoid all the faff mentioned above.

generalciao commented 2 years ago

Hi Robert, and thank you! Our camera is triggered on each 2P frame, so at least that is good to go :) We also have a background IR array similar to your link (thanks!) and your automatic on/off is a good tip, thanks.

I assume your IR background array operates at a lower wavelength than your 2P laser? Did you add a blocking filter on your camera lens to block the 2P wavelengths, so that all you see is what's illuminated by the IR background array? Or you have a hybrid external IR and intracranial 2P-IR illumination of the eye?

I guess I should just ask: is the pupil lighter or dark than the surrounding eye in your current images? And just to be sure - do you still use the ImageJ inversion step in your current/final setup, i.e. with the IR background array?

Thank you!

7rebor commented 2 years ago

Hi generalciao,

I think our IR illuminator is emitting broad bandwidth IR and we do have a blocking filter on the camera (this one: https://www.thorlabs.com/thorproduct.cfm?partnumber=FB850-40), it is a bit strange that we have a bandpass filter, doesn't help if you image below 890 nm, we image GCaMP at 920 nm though, so we block that light.

We only use the external illumination now, no inversion needed. I also realised that placing the light under the camera (at the same angle as the camera) was bad. If the animal moved its eye to a certain position, the retina would reflect the light directly in to the camera, causing an issue with the contrast and pupil size estimation. It would do this quite frequently during recording. I found that pointing the IR light down the animal's snout was a good idea, ideally at some height so that it doesn't get a shadow on the eye from the eyelids.

EDIT: pupil is darker in our current images