ray_intersect not working on Firefox & Ubuntu 22.04

[santerioksanen]

Hi, first of all thanks for a great library! Please forgive me, that I am not seasoned with either graphics programming, wasm or windowing systems in Linux, so this issue might be due to user error :)

I however ran into an when determining the 3d coordinate for the cursor projected on top of a desired object. I am using ray_intersect for calculating the point of collision. However on my system on Firefox 119.0.1, the intersection point is always the position of the camera. This seems to originate from the shader or somewhere else further down the chain, as adding logging to the functions shows that the rendered pixel is indeed exactly zero.

On Chromium the same setup works as expected with the intersection point being correctly on the surface of the geometry.

I am running Ubuntu 22.04 with X11 windowing system. My GPU is RTX 2080, and glxinfo | grep "OpenGL version" returns OpenGL version string: 4.6.0 NVIDIA 525.147.05

[asny]

Thanks for reporting! You're right, picking is not working in firefox, works fine in chrome and safari 🤷 Fortunately firefox shows a relevant warning, so shouldn't be too hard to fix. I'll do it as soon as I have a bit of time 🙂

[santerioksanen]

Thanks for confirming! I did some digging around and managed to get it to work on Firefox as well. My fix is ugly though, and I am not aware of the side-effects, so won't make a PR of this.

Anyway, as the error message WebGL warning: readPixels: Format and type RED/FLOAT incompatible with this R32F attachment. This framebuffer requires either RGBA/FLOAT or getParameter(IMPLEMENTATION_COLOR_READ_FORMAT/_TYPE) RGBA/FLOAT. suggests, Firefox would like the return type of readPixels to be RGBA/FLOAT.

So I modified core.rs, line 209 in format_from_data_type case 1 to return crate::context::RGBA as well.

After this I received another warning about too small buffer. This was easily solved by modifying render_target.rs, and multiplying data_size in read_color_partially with 4.

Hope this helps with the fixing :)

[santerioksanen]

Rendering with NormalMaterial might be affected by a similar issue in both Chromium and Firefox, at least according to my testing.

I am trying to read the normals by the following method:

pub fn camera_normals(
    context: &Context,
    position: Vec3,
    direction: Vec3,
    geometries: impl IntoIterator<Item = impl Geometry>,
) -> Vec3 {
    use three_d::core::*;
    let width = 15;
    let height = 15;
    let viewport = Viewport::new_at_origo(width, height);
    let up = if direction.dot(vec3(1.0, 0.0, 0.0)).abs() > 0.99 {
        direction.cross(vec3(0.0, 1.0, 0.0))
    } else {
        direction.cross(vec3(1.0, 0.0, 0.0))
    };
    let camera = Camera::new_orthographic(
        viewport,
        position,
        position + direction * 100.,
        up,
        0.01,
        0.0,
        100.,
    );
    let mut texture = Texture2D::new_empty::<Vec4>(
        context,
        viewport.width,
        viewport.height,
        Interpolation::Nearest,
        Interpolation::Nearest,
        None,
        Wrapping::ClampToEdge,
        Wrapping::ClampToEdge,
    );
    let normal_material = NormalMaterial {
        ..NormalMaterial::default()
    };

    let normals: Vec<f32> = texture.as_color_target(None)
    .clear(ClearState::color_and_depth(1.0, 1.0, 1.0, 1.0, 1.0))
    .write(|| {
        for geometry in geometries {
            render_with_material(context, &camera, &geometry, &normal_material, &[]);
        }
    }).read();

    let mut ret = Vec3 {x: 0., y: 0., z: 0.};

    for i in 0..width*height {
        ret.x += normals[(i * 4) as usize];
        ret.y += normals[(i * 4 + 1) as usize];
        ret.z += normals[(i * 4 + 2) as usize];
    }

    ret /= (width*height) as f32;
    ret * 2. - vec3(1. , 1., 1.)
}

, but the Firefox debug console shows a similar result as with the depth: WebGL warning: readPixels: Format and type RED/FLOAT incompatible with this RGBA32F attachment. This framebuffer requires either RGBA/FLOAT or getParameter(IMPLEMENTATION_COLOR_READ_FORMAT/_TYPE) RGBA/FLOAT..

Again if I apply the aforementioned changes, the errors go away.

Furthermore the depth and normals seems to be unstable to some degree, will attempt to dig further into these.

[asny]

I finally, after three months, got a bit of time to fix this. Ray intersect is working after 5d52c48. The problem is that it's not possible to read a float from each pixel on Firefox, but it is possible to read 4 floats from each pixel. So the fix in you're code is simply to replace let normals: Vec<f32> =.. with let normals: Vec<[f32; 4]> =... In the same commit, I've updated the documentation and added a panic on web if you try to read less than 4 values from a render target.

[asny]

Oh and by the way, it's not a good idea to read the normals to the CPU if you want your application to perform. It's generally a bad idea to read from the GPU to the CPU unless you really need it 🙂 It's probably more efficient (and easier) to compute the normals on the CPU if you need them there.

[santerioksanen]

Thank you for taking the time to fix this, and your instructions! :)

My goal for the normal reading is to step a few units back from the intersection point between pointer and object, so I am only reading the normals for a single pixel. Thus haven't at least noticed any performance issues. Good to know anyway if I run into issues on less performant hardware.

[asny]

No problem 🙂 Ah cool. Yeah, reading the normal of a single pixel to CPU should perform just fine.