mannk2
commented
2 years ago Hi there Jeremy. I have a few points to make regarding your thoughts on this.
Yes, people do tend to default to the 'image as fast as possible and deal with it later' method. I'm even one of those people. In the case where generating too much data is burdensome, I totally get what you mean. Imaging at 15Hz is probably fine for calcium imaging. Certainly even 10Hz is good enough depending on what you're doing. With that in mind, there are a few points to be made regarding the system (and other systems as well, as the principles should be the same on other scopes).
1) Resonant scanning speed cannot be changed. This is a result of the fact that the resonant scanner runs at a fixed frequency. The only thing one can do to alter that is to change the number of lines you scan (adding more takes longer, for example). Therefore when you're imaging at 30Hz in resonant mode, you cannot actually tell the scan head to slow down. You could, for example, take a single frame (~33ms), and blank the laser for 33ms, and then collect another frame. This would skip a frame, giving you 15Hz imaging, but would still end up with no change in noise. You can, however, simply use frame averaging. In this case, the system still scans at 30Hz, but averages frames before dumping them. This will reduce your noise and also your file size, and seems to be what you want to do.
2) reducing your X spatial resolution while in resonant mode (looks like you do 512x512?) can also reduce noise by changing the dwell time per pixel. As the scan head sweeps the sample, the PMT is sampled as many times as possible per pixel. By reducing X resolution, the scan head will sweep at the same speed of course, but you will average across more PMT samples per pixel, giving you better SNR (the 'enable multisampling' feature, which should be on, will report how many times a pixel will sample the PMT in resonant mode). This is a great way to reduce the size of your files and increase your SNR, as you're collecting fewer pixels. However, as with the temporal resolution, use the spatial resolution that your experiment requires. Maybe you don't actually need 512 X pixels just as you don't need 30 frames per second. It'll help with both issues here. This is also a post-hoc option, as you can reduce the spatial resolution of already-collected data by averaging neighboring pixels and will achieve the same outcome.
3) Slowing imaging rates in galvo mode is indeed possible, as the galvo scan heads can be controlled in this manner. You can select a dwell time that corresponds to a desired frame rate, but it's going to be on the slow side. Reducing scan speed here is similar to a frame average in resonant mode, except that you're doing pixel averaging. I'm not sure what method Dr. Sakamoto is using, but the affect of reduced scan speed on a Bruker system is to simply increase the number of samples taken from the PMT due to the longer dwell time for the laser on each pixel. This will not make the sample brighter, but it will make it less noisy. In situations where you have a very dim sample and want to increase the brightness, you can certainly sum pixels post-hoc. This doesn't get around the issue of generating more data than desired, but perhaps early in the pipeline every n frames can be summed and then turned into a downsampled t-series, just like with frame averaging. This is more of a niche application though for dim samples.
Please let me know if this helps at all. I really do appreciate the care with which you put together these posts.
-Kevin
jmdelahanty
At the moment, we have the microscope collect data as quickly as possible. In the Ultima Investigator's case, this happens to be quite close to 30FPS.
The end result of this is that our data in the end, while certainly usable and yields traces that are reasonable, is quite noisy.
This issue is being created based off of two things:
30 FPS Might Be Unnecessary
Among the reasons for performing scans at the fastest possible rate with our imaging include, from my memory:
An additional convenience of using a 30FPS rate is that the camera recording the face of the subject is also recording at a speed fast enough to capture much of the motion of the subject's face. Note that the camera takes an image each time the microscope does as it's triggered by the microscope's output TTL start of frame triggers.
Potential Drawbacks of Continuing this Behavior
There are multiple potential drawbacks to continuing this practice of fastest possible scan acquisitions.
Faster scanning speeds means noisier data
The way multi-photon scopes capture data in resonant scanning is by the rapid movement of multiple mirrors that direct a laser light path into extremely specific points in space on the order of microsecond precision. Each pixel in an image is acquired according to the dwell time specified in Prairie View's software. However long the dwell time is specified is how long the laser remains at a point in the field of view. As the laser stimulates each of these points, photons are emitted from the genetically encoded calcium indicators. These photons are received by the Photo-Multiplier Tube (PMT) Gallium Arsenic Phosphate (GaAsP) surface that is highly sensitive to light. This triggers a cascade down the PMT in which the electrons excited by the incoming photons are amplified exponentially as they are drawn down the tube by the high voltage supplied to the tube's surfaces. As these electrons are pulled through the tube, a current is generated which is then sampled by the Data Acquisition Card (DAQ). The software then turns the measurements from the card into a pixel intensity within the uint16 (meaning 16bit) range of numbers.
A consequence of moving as fast as possible through these points in the field of view (FOV) is that the system samples few photons from each point in space. As noted [here]() there are several kinds of noise that are introduced in these types of imaging sessions. If one samples only very quickly through each point in space, the relative amount of signal to noise will be smaller. In other words, if you sample each position only very quickly, you are collecting smaller amounts of real signal (biological fluorescence).
Although temporally smoothing things via a running average can be helpful in reducing the relative amounts of signal to noise and it does indeed produce images that are clearer in appearance, from what multiple people have told me it can be better practice to simply slow down the frame rate of the imaging session and acquire better SNR at experimental runtime.
One scientist online I briefly messaged with, Dr. Masayuki Sakamoto from Kyoto University had this to say:
Another scientist I messaged, Dr. Nuné Martiros (who seemed to know Kay and Romy interestingly!) had this to say:
There are other filtering steps that some programs/people have suggested to perform on the data as well but that can be part of a different issue somewhere else.
Faster scanning means bigger data
Simply put, the more samples you take the more datapoints you have! The more datapoints you have, the more data you have to stuff into a file! At the moment, a single channel raw binary from Bruker at 30Hz puts out approximately 75GB. If we were to run the scope at 15Hz we would divide that in half!
Currently an approximately 25 minute recording at 30Hz yields about 45k tiff images totaling about 22GB. We could end up with higher quality imaging that only has 11GB total for a given subject's recording day! The savings long term could be quite high if we modify this parameter and just collect what we need for imaging calcium indicators.
If we were to use Python and Dask to go to H5/Zarr with lossless compression those file sizes could be reduced down to just 7/8GB per recording! Note that 2-photon data doesn't compress very well because it's inherently noisy/there is shot noise that's always there.
Configurability of Scope Frame Rate
This is something that should be allowed to have configurations made for it inside the configuration file. Something as simple as:
"frame_rate" : 15in the configuration .json file could be how it's stored and implemented. In order for this to succeed, should we choose to do it, would be to take that frame rate in the file and then:
prairieview_utilsfunction that does this conversionprairieview_utilsfunction that updates the software's settings viasetstatecommands (need to ensure that this is possible, I believe it is)