How to use alpha?

[Rochet2]

I am trying to render triangles that use alpha. However, my tests indicate that alpha is not taken into account by the rendering pipeline. I am using euc 0.5.3

I took the example code and made it render two triangles on top of each other. One is red and one is blue. Changing the alpha value of the red or blue color does not change the output in any way.

Is there some setting I am missing? I did not stumble on any examples regarding alpha and could not find anything regarding it in the documentation. However, the following issue mentions alpha blending: https://github.com/zesterer/euc/issues/9

Here is my code, which is the triangle example modified.

```rust use euc::{buffer::Buffer2d, rasterizer, DepthStrategy, Pipeline}; struct Triangle; impl Pipeline for Triangle { type Vertex = [f32; 4]; type VsOut = (f32, f32); type Pixel = u32; // Vertex shader // - Returns the 3D vertex location, and the VsOut value to be passed to the fragment shader #[inline(always)] fn vert(&self, pos: &[f32; 4]) -> ([f32; 4], Self::VsOut) { ([pos[0], pos[1], 0.0, 1.0], (pos[2], pos[3])) } // Specify the depth buffer strategy used for each draw call #[inline(always)] fn get_depth_strategy(&self) -> DepthStrategy { DepthStrategy::None } // Fragment shader // - Returns (in this case) a u32 #[inline(always)] fn frag(&self, out: &Self::VsOut) -> Self::Pixel { let mut bytes = [255, 0, 0, 255]; // Red // println!("{:?} {}", out, out.1 == 1.0); if out.1 == 1.0 { // Changing the 4th component here between 0 and 255 for example does not change the output bytes = [0, 0, 255, 0]; // Blue } (bytes[2] as u32) << 0 | (bytes[1] as u32) << 8 | (bytes[0] as u32) << 16 | (bytes[3] as u32) << 24 } } fn main() { const W: usize = 800; const H: usize = 600; { let mut color = Buffer2d::new([W, H], 0); Triangle.draw::, _>( &[ // Big red triangle [-1.0, -1.0, 1.0, 0.0], [1.0, -1.0, 1.0, 0.0], [0.0, 1.0, 1.0, 0.0], // Small blue triangle [-0.5, -0.5, 0.0, 1.0], [0.5, -0.5, 0.0, 1.0], [0.0, 0.5, 0.0, 1.0], ], &mut color, None, ); let mut win = minifb::Window::new("Triangle", W, H, minifb::WindowOptions::default()).unwrap(); while win.is_open() { win.update_with_buffer(color.as_ref(), W, H).unwrap(); } return; } } ```

And here is an image of the result

[zesterer]

Unfortunately, the current master branch doesn't really support blending. However, thankfully, the new refactor branch does, and it's pretty much production-ready!

You can see that in the Pipeline trait there's a blend_shader function that you can implement to specify how the old pixel should be blended with the new fragment to create a new pixel: https://github.com/zesterer/euc/blob/refactor/src/pipeline.rs#L224

[Rochet2]

Tried it out now. Got the alpha to work fine I believe.

However, the refactor branch caused some triangles to be lost somewhere along the rendering process. I am having black cutouts here and there, especially in text, which likely consists of a large number of triangles. Half of a rectangle of the background is missing entirely.

Even if I render a solid color, the missing triangles are not there. The triangles I feed to the system are the same as on the default branch. I believe I managed to set the same settings as well. I use no backface culling (Disabled/None) in both branches and the depth is none/Empty::default().

Here is an example of the expected and actual. First with 0.5.3 and then with the refactor branch. Then with refactor branch using solid red color.

It is possible that the text is fuzzy because of incorrect color blending still though. But the missing triangle at the back is not clear why it would be missing. The expected result for the red coloring is that the entire area is a solid red color. There should be no black visible at all or any elements - just a solid red rectangle.

[zesterer]

What happens if you disable culling entirely? Do you still get the problems? Also, how is the text generated? Is it made of lots of small polygons, or a larger one with a texture?

[Rochet2]

What happens if you disable culling entirely?

How can I disable culling? I thought passing CullMode::None to render already did that, which is what I do.

// Old code
Triangle{...}.draw::<rasterizer::Triangles<(f32,), euc::rasterizer::BackfaceCullingDisabled>, _>(index_buffer, pixel.into(), None);
// New code
Triangle{...}.render(index_buffer, euc::CullMode::None, pixel.into(), &mut euc::Empty::default());

Also, how is the text generated?

Its made with larger one with a texture. So fuzziness is very likely caused by bad blending still. However, at least blending is partially working already. So main issue is the missing background triangle.

Tried to debug some: I tried rendering the same view with both branches and using a solid color. I printed the data that vertex shader is outputting and compared the output with diff tool on both versions. It seems that the data that comes into and goes out of the vertex shader is the same. There is the same amount of data printed for both branches and the same data is printed.

The fragment shader always produced red. I used a global variable to count the calls to fragment shader. On the default branch, the count was consistent between multiple starts of the program, which was expected. However, on the refactor branch the count between fragment shader calls on each start fluctuated, which is unexpected. Same occurs for blend shader function call count (naturally).

The global variable to count calls did not use mutex or similar, so if threading is used on the refactor branch, then the data is bogus.

[zesterer]

Do you have a minimal example I can take a look at locally? It might be an issue on my end that I've managed to miss while testing.

[Rochet2]

To be clear, I assume the text issue is caused by my code not being complete yet.

Here is code that tries to draw the background rectangle only. Added code for both branches and a picture of the results.

refactor branch (unexpected):

 ```rust use euc::{buffer::Buffer2d, rasterizer, Pipeline}; use minifb; use euc::{buffer::*, CullMode, Target}; use vek::*; struct Triangle { dimensions: [f32; 2], vertex_buffer: Vec<[f32; 2]>, } impl Pipeline for Triangle { type Vertex = u32; type VertexData = f32; type Pixel = u32; type Primitives = euc::TriangleList; type Fragment = f32; #[inline(always)] fn vertex_shader(&self, vertex_index: &Self::Vertex) -> ([f32; 4], Self::VertexData) { let vertex_xy = self.vertex_buffer[*vertex_index as usize]; let ret = [ 2.0 * vertex_xy[0] / self.dimensions[0] - 1.0, 1.0 - 2.0 * vertex_xy[1] / self.dimensions[1], 0.0, 1.0, ]; (ret, 0.0) } #[inline(always)] fn fragment_shader(&self, vs_out: Self::VertexData) -> Self::Fragment { 0.0 } fn blend_shader(&self, old: Self::Pixel, new: Self::Fragment) -> Self::Pixel { // RED (255 as u32) << 24 | (255 as u32) << 16 | (0 as u32) << 8 | (0 as u32) << 0 } } const W: usize = 800; const H: usize = 600; fn main() { let mut win = minifb::Window::new("Minimal example", W, H, minifb::WindowOptions::default()).unwrap(); while win.is_open() { let mut color = Buffer2d::fill([W, H], 0); // vertex positions [x,y] let vertex_buffer = vec![[0.5, 0.5], [223.5, 0.5], [223.5, 599.5], [0.5, 599.5]]; let index_buffer = [1, 0, 2, 2, 0, 3]; Triangle { vertex_buffer, dimensions: [W as f32, H as f32], } .render( index_buffer, euc::CullMode::None, &mut color, &mut euc::Empty::default(), ); win.update_with_buffer(color.raw(), W, H).unwrap(); } } ```

default branch (expected):

 ```rust use euc::buffer::Buffer2d; use minifb; use egui::{ color::*, emath::Rect, epaint::{Color32, Mesh}, Texture, }; use euc::{buffer::*, rasterizer, DepthStrategy, Pipeline, Target}; use image::{RgbImage, RgbaImage}; use std::path::Path; use vek::*; struct Triangle { dimensions: [f32; 2], vertex_buffer: Vec<[f32; 2]>, } impl Pipeline for Triangle { type Vertex = u32; type VsOut = f32; type Pixel = u32; #[inline(always)] fn vert(&self, vertex_index: &Self::Vertex) -> ([f32; 4], Self::VsOut) { let vertex_xy = self.vertex_buffer[*vertex_index as usize]; let ret = [ 2.0 * vertex_xy[0] / self.dimensions[0] - 1.0, 1.0 - 2.0 * vertex_xy[1] / self.dimensions[1], 0.0, 1.0, ]; (ret, 0.0) } #[inline(always)] fn get_depth_strategy(&self) -> DepthStrategy { DepthStrategy::None } #[inline(always)] fn frag(&self, out: &Self::VsOut) -> Self::Pixel { // RED (255 as u32) << 24 | (255 as u32) << 16 | (0 as u32) << 8 | (0 as u32) << 0 } } const W: usize = 800; const H: usize = 600; fn main() { let mut win = minifb::Window::new("Minimal example", W, H, minifb::WindowOptions::default()).unwrap(); while win.is_open() { let mut color = Buffer2d::new([W, H], 0); // vertex positions [x,y] let vertex_buffer = vec![[0.5, 0.5], [223.5, 0.5], [223.5, 599.5], [0.5, 599.5]]; let index_buffer = [1, 0, 2, 2, 0, 3]; Triangle { vertex_buffer, dimensions: [W as f32, H as f32], } .draw::, _>( &index_buffer, &mut color, None, ); win.update_with_buffer(color.as_ref(), W, H).unwrap(); } } ```

[zesterer]

Those triangles have an opposing winding order. If you change the index buffer to [0, 1, 2, 2, 0, 3], does that fix it?

Edit: I've just tried it myself and it seems to. This seems like a culling bug, I'll fix it now.

[zesterer]

I've fixed this issue in 3b23507, it seems to render correctly now. Thanks for pointing this out!

This also reminded me that I push the refactor branch on hold while waiting for min_const_generics to be stable. Now that this has happened, I can start working towards releasing it for you.

[Rochet2]

Now the background issue is fixed, but the text is missing half of each rectangle. (see last image of https://github.com/zesterer/euc/issues/20#issuecomment-849456443) Later on, I will send you the rest of the triangles when I have time to extract the data so you can maybe better debug the issue.

[zesterer]

To me, this looks like a blending issue. It appears that the source pixels aren't being smoothly blended with the target, they're being binary blended past a specific threshold.

[Rochet2]

Ah, here is what I meant. Here is an image of the current result with the latest fix. I have removed the background (that now works) to be able to show the rest of the triangles that are now not rendering. I am rendering a solid color.

The text should be rendering as rectangles as seen on the right side of the previous image seen below. Try to ignore the large triangle on the bottom left here and the images should look identical.

Both images are from refactor branch, first one is after the change to fix the background. But anyways, Ill send you the data later.

[zesterer]

It's still culling triangles? That's quite bizarre. I'll definitely be writing some tests for this once we've gotten to the bottom of this problem.

[Rochet2]

Here is the data. It has the index buffer and vertex buffer. You can replace the data of the buffers in the examples in https://github.com/zesterer/euc/issues/20#issuecomment-849747750 and run the code to see the results. Use latest refactor branch and euc 5.3. https://gist.github.com/Rochet2/3f55ae8bb58ce7e7b901dc3f9486ee39

The result should look like the two images in https://github.com/zesterer/euc/issues/20#issuecomment-850320461

[zesterer]

Thanks, I'll try to get to this over the weekend.

[Rochet2]

I did a git bisect on the original issue related to the background triangle not being rendered. The bisect directs to this commit being the one that introduced the issue: https://github.com/zesterer/euc/commit/a13bd8c87c20572cbc640d92bb44ffbcf18b24a3

The commit does not seem to include references to winding. So it is possible that the winding is fine and the problem is elsewhere. I would suggest maybe (partially) reverting the winding commit (https://github.com/zesterer/euc/commit/3b23507ce2cb0533bffee71665596bd56dc5207a) and investigating the bug introducing code a bit.

Here is bisect log

$ git bisect log
git bisect start
# good: [6f6ee20becd2f8ea99418f8555d1d45b6494aff0] Bumped version
git bisect good 6f6ee20becd2f8ea99418f8555d1d45b6494aff0
# bad: [9439ddc2462b9172f2e51d87758c5a070cb26dca] MSAA experimental
git bisect bad 9439ddc2462b9172f2e51d87758c5a070cb26dca
# good: [935316faaa0c2ec043e8713c43e8886c79b654af] Create rust.yml
git bisect good 935316faaa0c2ec043e8713c43e8886c79b654af
# good: [c4398f15efa2802dfc60daa9c30c67bcc3d56962] Parallelism support
git bisect good c4398f15efa2802dfc60daa9c30c67bcc3d56962
# bad: [ce3b830c1d6c9a65fb8e5a81758eb0e354f52fe0] Experimental MSAA support
git bisect bad ce3b830c1d6c9a65fb8e5a81758eb0e354f52fe0
# bad: [5371cff8f97305c4f906ff371ae9579a50f6520d] Performance improvements, new logo, better README info
git bisect bad 5371cff8f97305c4f906ff371ae9579a50f6520d
# bad: [a13bd8c87c20572cbc640d92bb44ffbcf18b24a3] Experimental MSAA
git bisect bad a13bd8c87c20572cbc640d92bb44ffbcf18b24a3
# first bad commit: [a13bd8c87c20572cbc640d92bb44ffbcf18b24a3] Experimental MSAA

Here is the code for that bad commit that I expect to render a single red rectangle, but instead renders a triangle.

```rust // a13bd8c use euc::{buffer::Buffer2d, rasterizer, Pipeline}; use minifb; use euc::{buffer::*, CullMode, Target}; use vek::*; struct Triangle { dimensions: [f32; 2], vertex_buffer: Vec<[f32; 2]>, } impl Pipeline for Triangle { type Vertex = u32; type VertexData = f32; type Pixel = u32; type Primitives = euc::TriangleList; // type Fragment = f32; #[inline(always)] fn vertex_shader(&self, vertex_index: &Self::Vertex) -> ([f32; 4], Self::VertexData) { let vertex_xy = self.vertex_buffer[*vertex_index as usize]; let ret = [ 2.0 * vertex_xy[0] / self.dimensions[0] - 1.0, 1.0 - 2.0 * vertex_xy[1] / self.dimensions[1], 0.0, 1.0, ]; (ret, 0.0) } #[inline(always)] fn fragment_shader(&self, vs_out: Self::VertexData) -> Self::Pixel { 0 } fn blend_shader(&self, old: Self::Pixel, new: Self::Pixel) -> Self::Pixel { // RED (255 as u32) << 24 | (255 as u32) << 16 | (0 as u32) << 8 | (0 as u32) << 0 } } const W: usize = 800; const H: usize = 600; fn main() { let mut win = minifb::Window::new("Minimal example", W, H, minifb::WindowOptions::default()).unwrap(); while win.is_open() { let mut color = Buffer2d::fill([W, H], 0); // vertex positions [x,y] let vertex_buffer = vec![[0.5, 0.5], [223.5, 0.5], [223.5, 599.5], [0.5, 599.5]]; let index_buffer = [1, 0, 2, 2, 0, 3]; Triangle { vertex_buffer, dimensions: [W as f32, H as f32], } .render( index_buffer, euc::CullMode::None, &mut color, &mut euc::Empty::default(), ); win.update_with_buffer(color.raw(), W, H).unwrap(); } } ```

[amPerl]

Just bumping this since I don't want to create a new issue for something that's technically only a problem on a wip branch. The last snippet shared by OP currently correctly produces a rectangle, but when you change the order of the indices you get a single triangle again, so it looks like it's still broken.

[zesterer]

I'd be interested to know whether this is still an issue on the refactor branch.

[amPerl]

Sorry, yes, that's what I meant by WIP branch. I'm running into this as of the latest commit of refactor.

My understanding is that in the rectangle example with rasterizer_config returning CullMode::None, I should be able to get a solid rectangle regardless of the order of indices within a single triangle.

While that snippet works as-is (adjusted for api changes), if I change the order of the first triangle's indices to be clockwise: 0, 1, 2 (full [0, 1, 2, 2, 0, 3]), I get a single triangle.

[zesterer]

Any chance you could post the snippet here? I can look into this tomorrow.

[amPerl]

Sure, it's just the code from https://github.com/zesterer/euc/issues/20#issuecomment-932974359 adjusted for API changes and with swapped indices on the first triangle:

use euc::{buffer::Buffer2d, CullMode, Pipeline, TriangleList};

struct Triangle {
    dimensions: [f32; 2],
    vertex_buffer: Vec<[f32; 2]>,
}

impl Pipeline for Triangle {
    type Vertex = u32;
    type VertexData = f32;
    type Pixel = u32;
    type Primitives = TriangleList;
    type Fragment = f32;

    fn rasterizer_config(&self) -> CullMode {
        CullMode::None
    }

    #[inline(always)]
    fn vertex(&self, vertex_index: &Self::Vertex) -> ([f32; 4], Self::VertexData) {
        let vertex_xy = self.vertex_buffer[*vertex_index as usize];
        let ret = [
            2.0 * vertex_xy[0] / self.dimensions[0] - 1.0,
            1.0 - 2.0 * vertex_xy[1] / self.dimensions[1],
            0.0,
            1.0,
        ];
        (ret, 0.0)
    }

    #[inline(always)]
    fn fragment(&self, _vs_out: Self::VertexData) -> Self::Fragment {
        0.0
    }

    #[inline(always)]
    fn blend(&self, _old: Self::Pixel, _new: Self::Fragment) -> Self::Pixel {
        // RED
        (255_u32) << 24 | (255_u32) << 16
    }
}

const W: usize = 800;
const H: usize = 600;

fn main() {
    let mut win =
        minifb::Window::new("Minimal example", W, H, minifb::WindowOptions::default()).unwrap();
    while win.is_open() {
        let mut color = Buffer2d::fill([W, H], 0);
        let vertex_buffer = vec![[0.5, 0.5], [223.5, 0.5], [223.5, 599.5], [0.5, 599.5]];
        let index_buffer = [0, 1, 2, 2, 0, 3];
        Triangle {
            vertex_buffer,
            dimensions: [W as f32, H as f32],
        }
        .render(index_buffer, &mut color, &mut euc::Empty::default());
        win.update_with_buffer(color.raw(), W, H).unwrap();
    }
}

[zesterer]

Right, I found the bug and pushed a fix. It was really quite subtle and depended on the fact that the y coordinates of the upper two vertices are the same, but all seems to work now.

[amPerl]

Thanks!! I can confirm my application (integration for egui, same as OP coincidentally, minus the textures and alpha blending) now produces the expected shapes.

[zesterer]

Woo, that's good to hear :)

Very cool that you're using egui with euc btw. Is this the sort of thing that could be generalised into an alternative backend for egui?

[zesterer]

By the way, the refactor branch includes a blend_shader function that you can use to implement alpha blending. I should probably add some sort of utility that does this for the common cases though.

[Rochet2]

integration for egui, same as OP coincidentally

Haha. Cool stuff. 😃 What I needed originally was a pure Rust renderer as well as UI framework as I was targeting WebAssembly and there was no openGL API or similar available in the environment I had. I think this was the only pure rust renderer I found at the time. Maybe still is. It was quite slow on Wasm to render things. I tried to compile euc with SIMD support at the time for Wasm, but I either failed or the code isnt made in away that can use SIMD (auto vectorizing is not supported by code or compiler maybe?).

btw. Is this the sort of thing that could be generalised into an alternative backend for egui?

I think so. IIRC the only things needed are texture support, alpha blending. Both are available, at least on the refactor branch. The shaders need to be coded (or translated from existing ones) for euc.

Take a peek at the integration part here, only the "paint" function needs to be coded https://docs.rs/egui/latest/egui/index.html#integrating-with-egui And shaders here, which are really simple https://github.com/emilk/egui/tree/master/crates/egui_glow/src/shader

The user would need to gather all the inputs and pass them to egui. Egui would produce the triangles. The euc backend (Paint function) could take in the triangles and textures, render it and output the end result. The user would then need to decide what to do with the rendered output and how to display it.

[zesterer]

Interesting stuff!

It was quite slow on Wasm to render things

I can imagine this being the case. Opportunities for compiler optimisations are even narrower on WASM than on other targets right now, and euc is primarily optimised for correctness and flexibility over cutting-edge performance. That said, I've done a fair bit of optimisation work on the refactor branch, so hopefully things should be much better there.