[Bug]: Gouraud shading for `plt.tripcolor` is slightly off

[spanag]

Bug summary

When plt.tripcolor(... shading='gouraud') is used, triangles are not interpolated exactly as they should, there is a small but visible difference at times. A simple way to see it is to draw a rectangle with high aspect ratio, where each end has a different color (same in both vertices of each end).

Code for reproduction

from matplotlib import pyplot as plt
# Make a figure with a gouraud shaded band
fig = plt.figure(figsize=[6.4, 4.8], dpi=100) # increase dpi for detail
z = .05 # half width. try also: .02, .01
plt.tripcolor([-1,-1,+1,+1], [-z,+z,-z,+z],  [0,0,1,1], 
              shading='gouraud', cmap='gray');
plt.xlim((-1,+1)); plt.ylim((-1,+1)); 
plt.axis('off'); fig.patch.set_facecolor('k')

# Create a bytes buffer to save the plot
import io, numpy as np; from PIL import Image
buf = io.BytesIO()
plt.savefig(buf, format='png') # format doesn't affect the result
buf.seek(0)
# Open the PNG image from the buffer and convert it to a NumPy array
image = np.array(Image.open(buf),dtype='float32')
# Close the buffer
buf.close()

plt.show() # after saving

# Inspect the RGB array
img = image[...,1]/255; h,w = img.shape
plt.figure(); plt.title('Brightness per pixel column')
for s in [0.3, 0.5, 0.7]:
    plt.plot(img[:,round(s*w)], '.');
plt.xlim((.504-z/1.9)*h,(.504+z/1.9)*h)
plt.show()

Actual outcome

The shading is different for the two triangles, when linear interpolation should have captured the gradient exactly. It is visibly discontinuous along the diagonal of the rectangle where the triangles meet.

Expected outcome

A smooth linear gradient should be shown.

Additional information

Numerical inspection shows that there is a steady difference in brightness between the triangles.

The effect also appears for different (non-rect) vertex positions, values, colormaps etc.
Interestingly enough, it vanishes as the aspect ratio approaches 1.

I wonder if the different number of vertices from the 'bright' side per triangle (1 or 2) causes the offset.

For comparison, this rectangle with a linear gradient is shown smoothly with no seam, with classic OpenGL:

float hh = -.05;
glBegin (GL_TRIANGLE_STRIP);
glColor3f (0.0f, 0.0f, 0.0f);   glVertex2f (-1.0f, -hh);
glColor3f (1.0f, 1.0f, 1.0f);   glVertex2f (+1.0f, -hh);
glColor3f (0.0f, 0.0f, 0.0f);   glVertex2f (-1.0f, +hh);
glColor3f (1.0f, 1.0f, 1.0f);   glVertex2f (+1.0f, +hh);
glEnd ();

Operating system

Ubuntu

Matplotlib Version

3.9.2

Matplotlib Backend

module://matplotlib_inline.backend_inline

Python version

3.11.4

Jupyter version

lab 4.0.5

Installation

pip

[timhoffm]

It seems that both directions are "off". The following picture has a high-resolution pcolormesh in the center region (between the red markers), which we can consider as ground-truth. above and below are two identical versions of the tripcolor. One can see that deviations happen towards the "tip" in both cases. I'm not an expert in gouraud shading. Either this is a bug or a fundamental property of the algorithm.

from matplotlib import pyplot as plt
import numpy as np
# Make a figure with a gouraud shaded band
fig = plt.figure(figsize=[6.4, 4.8], dpi=100) # increase dpi for detail
z = .2 # half width. try also: .02, .01
plt.tripcolor([-1,-1,+1,+1], [0, z, 0 , z],  [0,0,1,1], shading='gouraud', cmap='gray');
plt.tripcolor([-1,-1,+1,+1], [3*z, 4*z, 3*z, 4*z],  [0,0,1,1], shading='gouraud', cmap='gray');

X, Y = np.meshgrid(np.linspace(-1, 1, 500), [1.5*z, 2.5*z])
Z = X
plt.pcolormesh(X, Y, Z, cmap='gray')

plt.hlines([0.2, 0.6], -1, -0.95, color='r')

plt.xlim((-1,+1)); plt.ylim((0,+1)); 
plt.axis('off')

[jklymak]

I think the reason for Gouraud shading behaving this way is a lot more obvious if you have three different colours on each triangle vertex. The colour at any interior point is a linear interpolation of the colours at the vertices, so a triangle with two black and one white vertex will be darker most of the triangle than one with one black and two white.

https://en.wikipedia.org/wiki/Gouraud_shading https://en.wikipedia.org/wiki/Mach_bands

I'm actually going to take the liberty of closing this as it's well-understood.

[timhoffm]

I still believe there's something slightly off: Take this triangle:

As far as I understand gouraud shading, we can linearly interpolate in any direction. E.g. choose any point D on the line AB. Since A and B have the same color D has that as well. Any connection line D'C should yield the full cmap from dark to light. (As a corrollary, we have constant color in the vertical direction. - So far so good.

But I'd expect that the horizontal line EC should exhibit the same full cmap as a pcolormesh does. And since vertical colors in the triangle are constant, the triangle should completely blend into the pcolormesh in the background. Apparently, there's a slight difference on the lighter side. Mach bands are not an explanation here because they only exagerate differences, but I expect there to be no differences at all.

 from matplotlib import pyplot as plt
import numpy as np
fig = plt.figure(figsize=[6.4, 4.8], dpi=100) # increase dpi for detail

X, Y = np.meshgrid(np.linspace(-1, 1, 2000), [-1, 1])
Z = X
plt.pcolormesh(X, Y, Z, cmap='gray')
plt.tripcolor([-1,-1,+1], [-1, 1 , 0],  [0,0,1], shading='gouraud', cmap='gray');
plt.axis('off')
plt.ylim(-2, 2)
plt.show()

Note that I've sampled the meshgrid larger than the number of pixels in the x direction to exclude any positioning effects (is the of a mesh element taken from left / right edge)?

[timhoffm]

Additional note: I've enlarged to full screen, zoomed into the triangle tip (top 10-20%) and took a sceenshot:

We see vertical lines where the grey value each changes by one. This is expected because we only have 256 greyscale colors. What's not to be expected is the different color values in the mesh and tripcolor. In the mesh, the most right value is correctly (255, 255, 255). But in the triangle tip, the lightest value is (249, 249, 249)

Here I've addtionally adjusted, lightness and contrast of the screenshot to see the effect more clearly:

[timhoffm]

There seems indeed to be some color issue as the tip is 6 shades of gray away from the end of the expected color.

[timhoffm]

Additional note:

When pointing the tip upwards, the effect is basically gone - with adjusted contrast/brightness:

You still see the tip, but the borders are just half a step shifted. There can be various reasons for it, and I would count this still as equivalent.

[jklymak]

It is a three point linear interpolation, not two points. Again do with three colors and it will make more sense. You are just confusing the issue using two colors.

[spanag]

Please refer to the math of Gouraud shading: when shading between two vertices, the third one shouldn't matter. And hence the illusion of Mach bands is that the eye sees "discontinuity" when in fact there are no jumps in the shading (only its slope). Gouraud shading is not supposed to have discontinuities across triangles.

[timhoffm]

It is a three point linear interpolation, not two points. Again do with three colors and it will make more sense. You are just confusing the issue using two colors.

The "three point linear interpolation" is a linear interpolation on the side lines, followed by an interpolation across the triangle from two side points. Since everything is linear it does not matter in which order / from which intermediary side points you interpolate. I've chosen a geometry and (two) color values that makes it easy to reason about the results. Are you saying my argument is wrong? If so, can you quantitatively explain why the tip color should be darker in my example (or in the original example above)?

[jklymak]

To my understanding Gouraud shading in general is to calculate the colors at the vertex and then linearly interpolates into the interior:

The "three point linear interpolation" is a linear interpolation on the side lines, followed by an interpolation across the triangle from two side points.

I agree that is how Gouraud originally did the linear interpolation step in 1971 - where the linear interpolation all happened on horizontal scan lines. But there have been subsequent interpolation schemes proposed (eg https://authors.library.caltech.edu/records/f3dj3-8an27). I can't parse which one agg uses, but I agree with you and @spanag that it is not exactly the same as Gouraud's original. But I will still argue that does not make it "wrong".

If so, can you quantitatively explain why the tip color should be darker in my example (or in the original example above)?

Again, I agree that in the original algorithm, it should be the same. However, you can easily have an interpolation where it is not the same. eg x = (x1 / d1 + x2 / d2 + x3 / d3) * (d1 + d2 + d3), where di is the distance to the vertex from where you want to calculate x. That is my knee jerk reaction as to what the "correct" interpolation scheme should be. The agg code is a little too mysterious to be clear what we actually use.

[greglucas]

I tried the example with mplcairo and I don't think I see the artifact there (my eyes may be playing tricks on me). So, I wonder if this is only an Agg issue.

Some other notes:

• Saving to pdf maybe shows a slight triangle, but not quite as noticeable as the Agg png
• Saving to svg looks like there is another bug not picking up the triangle shading (mplcairo appears to save a good svg).
• Saving to ps also shows a really bad result (mplcairo is also not great and has a different artifact on the lines between the triangles)

[timhoffm]

From the little I understand in agg_span_gouraud_gray.h / agg_span_gouraud_rgba.h the algorithm seems close to the original (having upper and lower triangles and horizontal scanning).

I've tried to compare horizontal and vertical gradients.

fig, ax = plt.subplots(figsize=(8, 8))
ax.tripcolor([-1,-1, 0], [-1, 1 , 0],  [0, 0, 1], shading='gouraud', cmap='gray');
ax.tripcolor([1,1, 0], [1, -1 , 0],  [0, 0, 1], shading='gouraud', cmap='gray');
ax.tripcolor([-1, 1, 0], [-1, -1 , 0],  [0, 0, 1], shading='gouraud', cmap='gray');
ax.tripcolor([-1, 1, 0], [1, 1 , 0],  [0, 0, 1], shading='gouraud', cmap='gray');
ax.set(xlim=(-0.05, 0.05), ylim=(-0.05, 0.05))
ax.axis('off')

This yields:

and after brightness / contrast adjustment:

We can see:

1. The horizontal gradients are slightly darker towards the bright tip.
2. The left-pointing triangle has skewed segmentation boundaries.

(2.) Seems weird at first, but I suppose it's a discretization propertry of the algorithm. You can see that segments follow the slope of the left sides of the upper / lower triangle; e.g. ax.tripcolor([1, 1, 0], [0.5, -1 , 0], [0, 0, 1], shading='gouraud', cmap='gray') with adjusted contrast/brightness:

This likely comes from the fact that you start the scan line on the left side with a discretized color. Since the gradient is constant, the step to the next color happens at a constant x-offset from the left triangle side.

Summary

Overall, we have established that:

1. horizontal colors are not exactly correct. IMHO this counts as a but.
2. The segmentation boundaries following the triangle sides is a discretization effect. This is ok. It would be a feature to make the segmentation boundaries follow the actual gradient and not the triangle boundaries. I suppose this is possible if you do not discretize to colors during the interpolation on the left edges, which give the starting points for the scan line.

I would consider both topics as:

• difficulty=high (one has to understand the AGG C++ code)
• priority=low (The visual impression is mostly right, there are few applications that rely on exact color values)

This means, we as core developers won't work on it in the foreseeable future, but if somebody wants to dive into this, a contribution would be highly welcome.