Feature request: zsmooth parameter for Heatmapgl

[mmlanger]

the zsmooth parameter seems to not be implemented in Heatmapgl , while it is in Heatmap.

Could you add this in the future? By default Heatmapgl seems to use interpolation while I do like to turn it off.

[etpinard]

We're actually thinking about deprecating heatmapgl. We found that our non-webgl implementation performs just as well as the WebGL version and is way less bug-prone.

Could you tell us why you're using heatmapgl? Thank you!

[mmlanger]

For larger images, e.g. a megapixel, the webgl implementation seems to perform better in my jupyterlab sessions, hovering and especially zoom is way slower than in the non-webgl heatmap. Reaction times of UI elements in general seem to be better in the webgl version.

Example code for a notebook:

import numpy as np
import plotly.graph_objs as go

layout = dict(width=800, height=800)

res = 1000

x_space = np.linspace(-np.pi, np.pi, res)
y_space = np.linspace(0, 2*np.pi, res)
x_grid, y_grid = np.meshgrid(x_space, y_space)
img = x_grid * np.sin(y_grid) + np.sin(14*y_grid)

Heatmapgl:

fig = go.FigureWidget(layout=layout)
heatmap = fig.add_heatmapgl(x=x_space, y=y_space, z=img)
fig

Normal Heatmap:

fig = go.FigureWidget(layout=layout)
heatmap = fig.add_heatmap(x=x_space, y=y_space, z=img)
fig

I use plotly 3.1.1 and jupyterlab 0.33.11 from the anaconda distribution.

[alexcjohnson]

I notice a strange interpolation effect in heatmapgl, that makes it look as though there's a NW/SE correlation between neighboring data points. If I zoom in on https://codepen.io/etpinard/pen/mojEWY?editors=111 (which has totally random z, no correlation) I see: But if I change to heatmap, zsmooth: 'fast' it looks appropriately uncorrelated:

[CPrescher]

Hi,

is there any chance that the interpolation will be fixed, e.g. removed or possibile to change for heatmapgl? I also would like to use heatmapgl rather than heatmap since it performs much better with scientific 2d detector data with up to 3000x3000 pixels. The current strange interpolation makes it unfortunately unusable for this case. (we would like to be able to see individual pixels as rectangles when zooming in)

[LorenzoMinutolo]

I would like to use the heatmapgl fot the same reason of https://github.com/plotly/plotly.js/issues/2924#issuecomment-587359338. I have an insane matrix of detector to update in real time so offloading the work to a GPU would be very important. A "roll data along axis" feature would also be amazing.

[ordiology]

Scientists I'm working with are finding similar issues with the heatmapgl implementation -- they require a discretised version rather than a smoothed one as their data is recorded discretely. We have also found that the non-gl implementation doesn't work well with large datasets (the datasets are not excessively large and the heatmap renders ok but panning and zooming is too slow to update). The gl version of the heatmap works beautifully except for the smoothing issue.

I have implemented a discretised heatmap solution that uses a fork of Mikola Lysenko's original gl-heatmap2d code that is sourced in plotly. I didn't want to implement huge changes to the plotly source code as a first step, so I didn't implement a zsmooth option in the code. Instead, my implementation requires our scientists to npm link to my discretised version of the code in their stack rather than the originally sourced package. There is a minor edit to the plotly heatmapgl trace to increase the margins (as an extra patch is required in both dimensions for the discretised version).

My discretised version of the gl-heatmap2d code is here: https://github.com/ordiology/gl-heatmap2d/tree/discrete-heatmap2d

My modification to the plotly code can be found in my forked branch here: https://github.com/ordiology/plotly.js/tree/ordiology

The edits I made to the plotly code (which can really be considered a bit of a hack and not an optimally implemented solution) are described here: https://github.com/ordiology/plotly.js/blob/ordiology/README_LO.md

I am not experienced with contributing to open source software so I would need advice on how best to approach implementing this as an optimal solution. I would very much appreciate any help you can provide.

[nicolaskruchten]

@antoinerg is this related to https://github.com/plotly/plotly.js/issues/4842 or totally different?

[nicolaskruchten]

(The relationship I'm thinking of here would be: if we implement #4842 would we get heatmapgl-like performance for heatmap and keep smoothing control?)

[CPrescher]

Hi, just to clarify a bit more for me: in #4842 you talk about images. Would this then also work for a 2d array such as a detector image (similarly to what matplotlib is doing with imshow), or only for actual image formats?

[CPrescher]

Sorry, just read all the other issues and saw that this is the hole point of this feature...

[CPrescher]

So I tried out the current implementation of using the image trace as an alternative and it is unfortunately still way to slow. For it to be usable, there needs to be a responsive interaction with image dimensions of 2048 x 2048 px... Please see for example how responsive pyqtgraph is for this type of application

[nicolaskruchten]

Yes, we have not yet implemented #4842 so there is no change in performance yet :)

[CPrescher]

Ah ok, thank you for the information! Will there be a webgl implementation? I just tried out to use the image as a a texture on a plane in three.js and the performance is really really good in terms of interaction (zooming, panning) and also plotting speed. To me it looks like that webgl is very capable of downsampling the image when zoomed out and therefore provides much better performance.

[nicolaskruchten]

At the moment the only item on our roadmap is speeding up heatmap and image traces via #4842. It's my hope that the resulting performance improvements will meet the needs of folks who are looking for zsmooth on heatmapgl.

(edited to remove mention of sponsorship, as @ordiology has most of the solution built already, and it mostly needs integration)

[CPrescher]

Ok, thank you. This sounds good. I am looking forward to the new improved image trace.

With webgl version I did not mean the zsmooth for heatmapgl, but rather for "image", such as an "imagegl". It should be possible to just have a plane in webgl where the image is attached as texture and thus, benefit from the webgl speed.

I also do not think, that heatmapgl can be made fast enough for this type of application.

[nicolaskruchten]

With webgl version I did not mean the zsmooth for heatmapgl

OK, but that's what the current issue is about :) If you have other ideas or requests, let's please open up a separate issue for discussion.

[archmoj]

Closed via #4953.

[nicolaskruchten]

To anyone still subscribed to this issue: we'd like to understand the situations in which heatmapgl performs better than heatmap, if you could provide some examples of dataset size, interactions and OS/browser version please.

We are considering removing heatmapgl from future versions of Plotly.js and are trying to understand the impact of such a change.

[ordiology]

Sorry for the late reply @nicolaskruchten, but I can provide a little info about the limitations we found with heatmap. For a json dataset file size of 12M with data dimensions 627x6455 (over 4M z values) we found heatmap didn't cope well -- it took a fair while to load and didn't update when panning and zooming efficiently (it was quite painful). This was running on both Mac and Windows OS in modern browsers Chrome/Firefox/Safari/Opera and also from a dash server environment.

We find the heatmapgl implementation to be very responsive and, as we've increased the load to larger datasets, it has continued to remain so.

[CPrescher]

I have to state that I have given up to even use heatmapgl for interaction of large image data typically acquired at large scientific instruments, since this is still too slow. My goal was to have as fast interaction possibility as for example with pyqtgraph in desktop software. We do need not just panning and zooming of the image, but also fast change of colorscale (e.g. actually replotting). After a LOT of tinkering, I have come up with my own solution where i map the data to an image THREE.js image texture (this of course implies evenly spaced pixels) and use this within webglcanvas in plain d3. To my surprise this turned out to be even better performing than pyqtgraph.