KDE Rendering Experimental Program

[JoaoDell]

[UPDATED] Hello everyone, this PR contains an experimental implementation of Kernel Density Estimation calculations for a set of points. This uses some of fury's existing APIs and some straight vtk ones. This program may be a good example of shader usage and how to apply post-processing effects inside FURY. The result from running that should be:

Below, what is the idea here:

1. Render billboards with KDE calculations of points to an offscreen window.
2. Capture the offscreen window with a texture.
3. Render a second billboard, this time to the screen, with the texture of the prior rendered window, having a post processing effect applied to it (color map application).

In more details, what is done:

1. The offscreen and normal screen managers are created. The offscreen window is set to be for offscreen rendering with SetOffscreenRendering(True).
2. The parameters of the rendering are created: Random points are initialized, the sigma for the KDE is set, and the scale of the billboards is defined. The points are passed to the billboard actor as its centers.
3. shaders.shader_apply_effects() is used to tell OpenGL additive blending is desired for this code.
4. The billboards are rendered to the offscreen window.
5. After this render, the second billboard, the one that will receive the texture from the prior rendering, is created.
6. The colormap is chosen from matplotlib.colormaps and passed to the billboard as a texture named "colormapTexture" with the function colormap_to_texture()
7. The window is captured as a texture and passed to the second billboard with window_to_texture()
8. The billboard is then finally rendered to the screen.

Some minor details that may be useful for a better understanding of it:

• The billboard actor got a set o shaders for the KDE calculation. In its declaration, a function that calculates the non-normalized (yet, I intend to implement a normalized version) KDE as: $KDE(x, \sigma) = exp(-\frac{1}{2}(\frac{x}{\sigma})^2)$.
• For all of that to work, I have created some functions to ease the work, functions that may be even used later in other FURY applications, so I wonder it would be a good idea to make a separate small PR for them.

[skoudoro]

thank you for your feedback. we will look at all this links.

[codecov[bot]]

Codecov Report

Merging #804 (b611368) into master (3d53e48) will decrease coverage by 0.05%. The diff coverage is n/a.

:exclamation: Current head b611368 differs from pull request most recent head bf2b91a. Consider uploading reports for the commit bf2b91a to get more accurate results

@@            Coverage Diff             @@
##           master     #804      +/-   ##
==========================================
- Coverage   84.43%   84.38%   -0.05%     
==========================================
  Files          43       43              
  Lines       10274    10274              
  Branches     1393     1393              
==========================================
- Hits         8675     8670       -5     
- Misses       1239     1243       +4     
- Partials      360      361       +1     

see 1 file with indirect coverage changes

[JoaoDell]

After a pair with @devmessias, we made some progress on the FBO setup issue. Now, the FBO is sucessfully generated, and it seems ready to be drawn to. Apparently, the issue was that the context needed to have the window interactor initialized with ShowManager.initialize() before the it was set to the FBO. I will prepare a better explaination and investigate further on why is that soon.

[JoaoDell]

Hey @devmessias thanks for pointing those out, I have just updated this PR with my current progress:

1. I could manage to stabilize the shader issues and now they seem to be working.
2. I have cleaned better the code itself to make it simpler.
3. I am being able to render simple colors to the billboard screen, however, I cannot seem to be able to use the color attachment as a texture after drawing to it offscreen on the FBO. When I activate the color texture to render to the screen, it outputs a pitch black render. I think that is due to my lack of enough understandment on how vtkFBOs and vtkTextureObjects work and get activated, will look better into it.

Maybe @filipinascimento could have a better idea of what is happening as well.

[JoaoDell]

Hey everyone, I just updated and completely refactored this PR as, as well as it is base program, as it went on a slight different direction in comparison to its original intetion. Everything seems to be working now, my only question is if it would be better if I closed this PR and opened a new one with a new branch containing only the recent commits, as the older ones have been completely replaced, or should I rebase it, I don't know. What should I do?

Anyway, this version is ready for review 👍

[JoaoDell]

@ganimtron-10 I have fixed and updated most of your recommendations. Indeed, typedhint isn't a FURY standard, but I will wait Filipi and Serge comments on that to change anything. Thanks for the comprehensive review!

[JoaoDell]

Also, I just realised there is a native colormap generator in fury.colormp named create_colormap() so I just updated it with that.

[JoaoDell]

Okay @skoudoro, I will be looking into the existing examples to understand how I should explain and detail this one so we can try using it as an example then, thanks for the opinion!