heatmaps get aliased

[ssfrr]

MWE:

using Makie

data = zeros(500,500)
data[(-100:10:100).+250,(-100:10:100).+250] .= 1
heatmap(data, show_axis=false)

which displays:

This is in the Atom plot pane, but displaying in the OpenGL window behaves similarly. Resizing the window results in different pixels turning on and off.

I ran into this also when displaying images that scroll (in my case audio spectrograms), where the images appear to "shimmer" as they scroll.

Ideally the output pixels would be computed from a low-passed version of the original image data to prevent the aliasing. I'm not sure if there's GLSL functionality built-in for this, but one option would be to have the fragment shader for each pixel use a window surrounding it, rather than just the single interpolated value. I can try tweaking the intensity shader to see if I can improve the behavior.

You can see the un-aliased version by setting the output resolution as the same as the matrix (using the OpenGL window backend):

data = zeros(500,500)
data[(-100:10:100).+250,(-100:10:100).+250] .= 1
scn = Scene(resolution=(500,500))
hm = heatmap!(scn, data, show_axis=false)

This is on current Makie (ee78f7c6) and AbstractPlotting (537997a7) master.

[c42f]

Fixing this is probably as simple as turning on mipmap=true when creating the Texture from the image data which causes glGenerateMipmap to generate a multiresolution pyramid. Then set something like minfilter=:linear_mipmap_linear and let the GPU texture sampler do the interpolation.

For large size reductions of large textures this should give quite good performance with reasonable quality.

[SimonDanisch]

@c42f suggestion changes the image to: Which seems wrong? Pixel perfekt: With black white colormap it seems quite perfect though: When zooming into the pixel, it explains the color shift for the mipmap version:

[ssfrr]

When you downsample the image, each pixel in the downsampled image ends up being an average of some region surrounding it, so it’s not wrong that the peak values in the downsampled image aren’t the same as the original.

[SimonDanisch]

I wonder how to best add this... Something very easy would be, to make sure heatmap(Sampler(data; mipmap=true, ...)) works correctly. That seems like a good option, considering that we want to use the Sampler type more often going forward, and that we don't have to add a lot of new options to heatmap. On the other hand, we started wanting to have correctness first and performance second, without removing any fast paths. But, especially heatmap is often used for fast updating, large data, which must be slowed down quite a bit by generating mipmaps for every update, and if that works slowly, people usually dont seem to investigate why things are slow and just take away that Makie is too slow for the task -.-