Reading screen pixels?

[Luminoth]

Apologies for so many questions, I'm part of a book club that's running through Nature of Code and I'm trying to do the examples in Rust using your crate. :)

I'm curious if there is something equivalent to loadPixels() and updatePixels() in Processing / p5js? I see ways to set the Screen framebuffer from a texture, but I'm not sure if there is a way to read the framebuffer out to a texture so that it can be modified and re-applied? Something that would allow for the equivalent of this:

function setup() {
  createCanvas(640, 360);
  d = pixelDensity();

  loadPixels();
  const noiseScale = 5.0;
  for (let y = 0; y < height; y++) {
    const ny = y / height * noiseScale;
    for (let x = 0; x < width; x++) {
      const nx = x / width * noiseScale;
      let c = color(noise(nx, ny) * 255);
      for (let i = 0; i < d; i++) {
        for (let j = 0; j < d; j++) {
          const index = 4 * (y * width * d * d + i * width * d + (x * d + j));
          pixels[index] = red(c);
          pixels[index + 1] = green(c);
          pixels[index + 2] = blue(c);
          pixels[index + 3] = alpha(c);
        }
      }
    }
  }
  updatePixels();
}

Or maybe the right thing for me to do in that example is start with a fresh texture, modify it, and then apply that to the framebuffer (since it doesn't care about the original pixel state)?

I was also curious if there is a way to get at the underlying MVP matrix. A lot of the examples in the Nature of Code book use pixel coordinates, similar to Processing / p5js, which is easy enough to convert to what I believe is GL coordinates with this crate? The issue I have, however is when something like a translate() happens and my simple coordinate mappings get thrown off. If not, is the best answer to that to just track my own matrix and use screen.apply_matrix() before I draw?

[rennis250]

No worries! Thanks for the questions! You are inspiring me to get back into all of this. I have been really the only user of this crate for personal projects for years now. You might run into a few rough spots if using this to do everything in the Nature of Code, but it should be okay for most of it. I like your adventurous approach :-)

So, I do not have the framebuffer access exposed, but that should not be hard to enable in a Processing-esque way.

This crate uses normalized device coordinates (NDC) with center of the screen as the origin, right being positive x, and down being positive y, as I find that to be a more natural way to think about things that aligns with geometry lessons, while still following some computer graphics conventions. Two of my colleagues once asked that I include a way to specify coordinates directly in pixels, since that is important for our work. Now that you also mention it, I will add that in.

The underlying MVP matrix is also currently not exposed, but could be done. I never thought to do this, because Processing/p5.js don't do this, as far as I can tell, and I did not have a need for it yet. But, I am happy to explore the possibilities outside the p5 framework. While heavily influenced by p5, I do think some things could be done better.

I apologize that I was not able to upload the other new code this weekend (mostly just some minor bug fixes), like I promised, but work had to take some priority.

[Luminoth]

Oh no worries at all! It hasn't been too hard to work around most of the differences so far and I totally appreciate you considering those changes! I'm happy it can provide some inspiration :D It's honestly been a ton of fun working through the book examples with your crate, I'd have been pretty bummed if it wasn't around.