PBRT Fluorescence plugin documentation

[trissim]

There currently isn't any documentation or examples on how to use the the Fluorescence plugin for the PBRT rendering engine. Providing some example code / configurations using a neuron mesh exported from NeuroMorphoVis would be very useful.

I am currently writing a python pipeline that uses swc files from neurons generated in simulated neurite outgrowth generated by the DeNSE https://github.com/SENeC-Initiative/DeNSE package and then generates meshes from the generated swc using NeuroMorphoVis. The last step of the pipeline is to generate synthetic fluorescence microscopy images from the mesh files.

If anyone can provide me with some guidance to get started using the plugin, I would be glad to contribute documentation and example code and based on this pipeline.

[NadirRoGue]

Hello,

Both the PBRT rendering engine and the Fluorescence plugin have been recently added and are still work in progress. The code works, but testing and documentation are on the process of being done.

Being said that, an initial documentation can be found as python method on its API (python/plugin/fluorescence.py). Nonetheless, the plugin works by given the engine volumes, thus the meshes should be converted into volume before being feed into Brayns.

To start using the plugin, brayns should be started as usual, but specifying the fluorescence plugin to be loaded: --plugin braynsFluorescence

And the pbrt-v2 engine, which includes the integrators that support fluorescence --engine pbrtv2

Once brayns is up and running, interacting using python you will need to: 1) Make brayns use the florescence volume integrator of your choice:

from brayns import Client
import sys

brayns = Client("localhost:5000")

spp = 16

if len(sys.argv) > 1:
  spp = int(sys.argv[1])

renderParams = brayns.SamplerrendererRendererParams()
renderParams.surface_integrator_class = "path"
renderParams.volume_integrator_class = "singlefluorescence"
renderParams.pixelsamples = spp
brayns.set_renderer_params(renderParams)

2) Load a volume with the appropiate parameters

from brayns import Client, Fluorescence
import sys

brayns = Client("localhost:5000")
fluorescence = Fluorescence(brayns)

fluorescence.add_fluorescent_volume(fex=..., 
                                    fem=..., 
                                    epsilon=..., 
                                    c=..., 
                                    yieldv=...,
                                    gf=..., 
                                    p0=(0, 0, 0), 
                                    p1=(1, 1, 1), 
                                    name="My fluorescent volume")

3) Position the camera where you wish and let it render

However, I highly encourage you to wait until we have advance further and have a proper documentation and intensive testing of the engine and the plugin.

[marwan-abdellah]

@trissim

The following statement is not true.

The last step of the pipeline is to generate synthetic fluorescence microscopy images from the mesh files.

To generate a fluorescence image, yo need to create a fluorescent volume tagged with the spectral and optical properties. For further information, please refer to that link

[trissim]

@NadirRoGue Thank you for the boiler plate code. This should be enough to get me started. Since the plugin lacks testing and documentation, are there any tests I could write while I experiment with the plugin? I really like this project and would be glad to contribute if any way.

@marwan-abdellah Thank you for the clarification. Which file format should the volume files be in? Could you provide a sample volume as reference? I am thinking of writing code that would generate fluorescently labeled volumes from mesh files to facilitate automation of fluorescence image rendering. I would add options of where to add the fluorescent probes, such as in the cell membrane, cytoplasm or microtubules. Is this a feature that you would consider adding if I provided the code?

[marwan-abdellah]

@trissim There is already a tool to automate the generation of the fluorescent volumes even for different components from morphologies and meshes. But this tool is only for BBP, it hasn't been open sourced yet. Are you at BBP?

[trissim]

@trissim There is already a tool to automate the generation of the fluorescent volumes even for different components from morphologies and meshes. But this tool is only for BBP, it hasn't been open sourced yet. Are you at BBP?

@marwan-abdellah I am not at BBP, but I am interested in contributing to your projects. I am currently a first year Masters student at McGill University at the Fournier lab ( https://fournierlab.mcgill.ca/ ) I am working on deep learning approach to trace neurites from neurons grown in cultures using b-splines. Having access to this tool for automated volume generation would be of great help. You can take a look at my neuron_splines project repo that I will be updating later this week (I have made a lot of changes since the last commit). We can setup a meeting to discuss things further if you are interested in collaborating or assigning me some tasks to help contribute.

[NadirRoGue]

Documentation for the fluorescence plugin is available at https://brayns.readthedocs.io/en/latest/api_fluorescence.html

Closing this issue. For any further problem, open a new issue, please.