Actor subclasses [Enhancement]

[paulbrodersen]

I was planning to overhaul the documentation for the Actor subclasses. However, this turned out to be more complex than anticipated, so that I have now run out of the time that I am able to contribute for the moment. Below hence three items:

1. A text dump of the documentation that I have already written.
2. Some general suggestions on how I would structure the documentation.
3. Comments on the actor subclasses code.

I hope some of these will be useful.

A actors.md draft

Objects in 3D renderings are computationally represented as sets of 3D coordinates that form dense meshes, which delineate the surface of the object. Standard neurobiological data formats, however, typically do not specify the surface of their objects directly. For example, an SWC file specifies the morphology of a neuron as a graph of segments, with each segment being represented as a truncated cone with a start point, a diameter at the start point, an end point, and a diameter at the endpoint. So while this specification clearly implies a surface, it does not contain any explicit surface information. Actors in brainrender are classes that implement this transformation, i.e. they map user-provided numerical data to vedo Mesh objects, which can then be rendered.

Available actors are:

• Point(pos, name, radius, color, alpha, res): render a (anterior-posterior, dorsal-vental, medial-lateral) point coordinate as a sphere with a specified name, radius, color, opacity, and resolution. The name is a string used as a handle, when modifying the scene. A color can be a valid matplotlib or VTK color specification, which includes the standard named web colors derived from the CSS Color Module Level 3 standard, hex codes, and RGB tuples. The opacity alpha is a float between 0 and 1. The integer resolution determines the density of the created mesh. Higher densities result in smoother surfaces (but are computationally more resource intensive).

from brainrender import Scene
from brainrender.actors import Point

s = Scene()
s.add(Point((1000, 2000, 3000), radius=100, color="blue",      alpha=0.33, res=20))
s.add(Point((4000, 5000, 6000), radius=200, color="#ff0000",   alpha=0.66, res=10))
s.add(Point((7000, 8000, 9000), radius=400, color=(0, 255, 0), alpha=1.,   res=5))
s.render()

• Points(data, name, colors, alpha, radius, res): convenience class to render multiple Point actors at once. The data argument is either an (N, 3) numpy array or a filepath. radius and colors (plural!) are single floats and valid color specifications, respectively, or lists thereof matching the length of the coordinate array. Due to limitations in upstream packages, however, radius and colors cannot both be sequences. Other parameter specifications are as for Point

import numpy as np
from brainrender import Scene
from brainrender.actors import Points

s = Scene()

# supported: single radius, single color
s.add(Points(np.array([[1000, 2000, 3000], [4000, 5000, 6000]]), radius=100, colors="blue"))

# supported: multiple radii, single color
s.add(Points(np.array([[1000, 2000, 3000], [4000, 5000, 6000]]), radius=[50, 100], colors="blue", alpha=1))

# supported: multiple colors, single radius
s.add(Points(np.array([[1000, 2000, 3000], [4000, 5000, 6000]]), radius=100, colors=[blue", "red"], alpha=1))

# not supported: multiple radii, multiple colors
# s.add(Points(np.array([[1000, 2000, 3000], [4000, 5000, 6000]]), radius=[50, 100], colors=["blue", "red"], alpha=1))

# supported: import coordinates from .npy file
s.add(Points("/path/to/coordinates.npy", radius=100, color="blue"))

s.render()

• Line(coordinates, color, alpha, linewidth, name): render an (N, 3) numpy array of (anterior-posterior, dorsal-vental, medial-lateral) point coordinates as a line. linewidth is a float; other parameter specifications are as for Point.

  import numpy as np
  from brainrender import Scene
  from brainrender.actors import Line
  
  s = Scene()
  s.add(Line(np.array([[1000, 2000, 3000], [4000, 5000, 6000]]), linewidth=10))
  s.render()

• Volume(griddata, voxel_size, cmap, min_quantile, min_value, name, as_surface): render volumetric data (e.g. gene expression) from a numpy array or from a .npy file.

• Neuron(data, color, alpha, neurite_radius, soma_radius, name): render neuron morphology data from a MorphoNeuron object (e.g. instantiated with morphapi) or from a .swc file.

• Streamlines

• Cylinder

• Ruler

General comments on the documentation

The documentation currently is a weird in-between of a reference and a tutorial, which makes it unhelpful to both audiences, experienced developers and first-time users. Its structure strongly mirrors the structure of the code. However, while it compiles an inventory of available classes and methods, it lacks sufficient detail to be a useful reference. The style is colloquial but it contains no examples and the order in which material is covered doesn't mirror a typical workflow, so it's not a particular useful tutorial either.

To address this tension, I would split the documentation in two: a reference, preferably compiled from overhauled doc-strings, and a guided tour with many, simple examples that covers the topics in the order in which a first time user needs to know about them. For the latter, I would suggest the following structure:

0. Installation

1. Instantiating a `Scene` with a reference atlas

1.1 Basic invocation
1.2 Available atlases
1.3 Highlighting brain regions

2. Adding experimental data

2.1 Registering data to the reference atlas
2.2 Adding actors to a scene
2.3 Available actors
2.4 Manipulating actors

3. Rendering & exporting

3.1 Basic invocation
3.2 Cameras
3.3 Export to PNG or HTML
3.4 Animations

4. Known issues

4.1 Notebooks
4.2 IPython

Code comments

General comments

Actor arguments are in dire need of standardisation:

• argument order varies widely between actors
• most variables have proper names but some don't (res, s)
• some actors have a res parameter even though probably most could/should

• there are subtle naming differences and inconsistencies:

  Point has the color argument, but Points has colors even though colors can be a single color specification. Both have the argument radius, which, however, can also be a sequence in Points (albeit an undocumented feature)!

Actor-specific

Neuron

IIRC, the SWC file standard supports neurite segment start and end thicknesses and soma diameters. As far as I can tell, the implementation, however, does not appear to expect/use that information. If that assessment is mistaken, neurite_radius and soma_radius should probably be called something else (default_neurite_radius, etc?).

Volume

The min_quantile/min_value argument pair is very difficult to communicate. I would either have a single threshold argument (and let the user compute percentiles). Or, I would follow vedo and have vmin/vmax arguments, which would pair better with cmap and additionally lean on people's familiarity with matplotlib.

Streamlines

There is no documentation anywhere on what the basic streamlines data structure entails. Even the examples/streamlines.py file downloads data from the ABA. This makes it quite difficult to document the class.

ruler & ruler_from_surface

Why are these functions when all other actors are classes?

Also, I would remove the unit and unit_scale arguments and hardcode "micrometers" as unit. Currently, coordinates are implicitly in micrometers, but the default unit is the empty string, and the unit_scale is 1, neither of which communicates this to the user. I would remove the arguments as everything about them is confusing and near-impossible to communicate.

Cylinder

Currently, cylinders are hard-coded to be vertical (unless I am missing something). Making the second argument another 3d coordinate (instead of Scene.root) would make this class a lot more flexible and easier to explain.

[adamltyson]

Thanks @paulbrodersen, much appreciated! Hopefully we will get to improving all of this sometime soon.

[paulbrodersen]

Sorry that I have to bail out like this (and pollute your issue tracker in the process with whatever this is), but I have a busy couple of weeks ahead. brainrender is a neat library, it just needs a little bit more documentation. Best of luck!