napari-sim-processor

A Napari plugin for the reconstruction of Structured Illumination Microscopy (SIM) with GPU acceleration (pytorch/cupy if installed). Currently supports:

• conventional SIM data with a generic number of angles and phases (typically, 3 angles and 3 phases are used for resolution improvement in 2D, but any combination can be processed by the widget)
• hexagonal SIM data with 7 phases, as used in this publication.
• 3D SIM, for resolution enhancement in three dimensions. This is available in the 3dSIM branch

The SIM processing widget accepts image stacks organized in 5D (angle,phase,z,y,x).

The reshape widget can be used to easily reshape the data if they are not organized as 5D (angle,phase,z,y,x).

For 3D stacks (raw images) with multiple z-frames, a batch reconstruction method is available, as described here.

Syntetic raw-image stacks of Structured Illumination Microscopy can be easily simulated using the napari SIMulator pluging.

This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

Installation

You can install napari-sim-processor via pip:

pip install napari-sim-processor

To install latest development version :

pip install git+https://github.com/andreabassi78/napari-sim-processor.git

Usage

1) Open napari.

2) Launch the reshape and sim-processor widgets.

3) Open your raw image stack (using the napari built-in or your own file opener).

4) If your image is ordered as a 5D stack (angle, phase, z-frame, y, x) go to point 6.

5) In the reshape widget, select the actual number of acquired angles, phases, and frames (red arrow) and press Reshape Stack. Note that the label axis of the viewer will be updated (green arrow).

6) In the sim-reconstruction widget press the Select image layer button. Note that the number of phases and angles will be updated (blue arrow).

7) Choose the correct parameters of the SIM acquisition (NA, pixelsize, M, etc.) and processing parameters (alpha, beta, w, eta, group):

• w: parameter of the Weiner filter.
• eta: constant used for calibration. It should be slightly smaller than the carrier frequency (in pupil radius units).
• group: for stacks with multiple z-frames, it is the number of frames that are used together for the calibration process.

For details on the other parameters see here.

8) Calibrate the SIM processor, pressing the Calibrate button. This will find the carrier frequencies (red circles if the Show Carrier checkbox is selected), the modulation amplitude and the phase, using cross correlation analysis.

9) Click on the checkboxes to show the power spectrum of the raw image (Show power spectrum) or the cross-correlation (Show Xcorr), to see if the found carrier frequency is correct.

Napari viewer showing the power spectrum of the raw stack. The pupil circle is in blue. A circle corresponding to eta is shown in green.

Napari viewer showing the cross-correlation of the raw stack. The red circles indicate the found carrier frequencies

10) Run the reconstruction of a single plane (SIM reconstruction) or of a stack (Stack reconstruction). After execution, a new image_layer will be added to the napari viewer. Click on the Batch reconstruction checkbox in order to process an entire stack in one shot. Click on the pytorch checkbox for gpu acceleration.

Napari viewer with widgets showing a pseudo-widefield reconstruction

Napari viewer with widgets showing a SIM reconstruction

GPU processing

The underlying processing classes will use numpy (and FFTW if available) for its calculations. For GPU accelerated processing you need to have either the PyTorch (tested with torch v1.11.0+cu113) or the CuPy (tested with cupy-cuda113 v10.4.0) package installed. Make sure to match the package cuda version to the CUDA library installed on your system otherwise PyTorch will default to CPU and CuPy will not work at all.

Both packages give significant speedup on even relatively modest CUDA GPUs compared to Numpy, and PyTorch running on the CPU only can show improvements relative to numpy and FFTW. Selection of which processing package to use is via a ComboBox in the napari_sim_processor widget. Only available packages are shown.

Other than requiring a CUDA GPU it is advisable to have significant GPU memory available, particularly when processing large datasets. Batch processing is the most memory hungry of the methods, but can process 280x512x512 datasets on a 4GB GPU.

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the BSD-3 license, "napari-sim-processor" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.