gpujs / gpu.js

GPU Accelerated JavaScript
https://gpu.rocks
MIT License
15.04k stars 646 forks source link

Render loop that uses the results of previous render tend to glitch out. #756

Open rgembalik opened 2 years ago

rgembalik commented 2 years ago

What is wrong?

I am trying to write a continuous simulation loop on canvas, using gpu.js. The idea is to:

  1. Generate an initial w * h * 4 array (Uint8ClampedArray to be more specific)
  2. Write a gpu.js kernel, that processes that data and outputs result with this.color
  3. return the data with getPixels() and override the initial data
  4. pass the data returned in step 3 to the next simulation
  5. repeat steps 3-5 in a render loop

However:

Here is the recorded result: gpu js-failed-simulation

Where does it happen?

In the browser. (chrome 101.0.4951.67)

How do we replicate the issue?

Here is a sample I wrote:

<!-- index.html -->
...
<canvas id="daCanvas" width="512" height="512"  >
...
// index.js
const canvas = document.getElementById('daCanvas');
const gl = canvas.getContext('webgl2', { premultipliedAlpha: false });

// 1.
let map = new Uint8ClampedArray(canvas.width * canvas.height * 4);

const gpu = new GPU({
    canvas,
    context: gl,
});

// 2.
const simulate = gpu.createKernel(function(mapData) {
   if(Math.random() > 0.95) {
        this.color(
            Math.min(1, mapData[this.thread.x][this.thread.y][0] / 255 + 0.3),
            Math.min(1, mapData[this.thread.x][this.thread.y][1] / 255 + 0.3), 
            Math.min(1, mapData[this.thread.x][this.thread.y][2] / 255 + 0.3), 
            1
        );
    } else {
        this.color(
            mapData[this.thread.x][this.thread.y][0] / 255, 
            mapData[this.thread.x][this.thread.y][1] / 255, 
            mapData[this.thread.x][this.thread.y][2] / 255, 
            1
        );
    }
})
.setOutput([canvas.width, canvas.height])
.setGraphical(true);

const render = () => {
    // 3.
    simulate(GPU.input(map, [canvas.width, canvas.height, 4]));
    // 4.
    map = simulate.getPixels();
}

// 5.
// Were using a nice slow interval for gentle simulation. Resutls for reqquestAnimationFrame loop are similar, just faster
setInterval(() => {requestAnimationFrame(render);}, 1000)

How important is this (1-5)?

It's just an experiment a'la the game of life, so let's say 2 (I take my hobby experiments seriously, but let's be honest - I am just playing with the tech).

Expected behaviour (i.e. solution)

I expected, that the image would start black, and then with each iteration, some pixels would become lighter and lighter until the image is entirely white.

Other Comments

I was trying this with several different approaches (separate kernel for processing, separate for rendering, different operations within the kernel etc), but the result was always glitchy. Even If I had just a loop that always draws whatever is in mapData and then would execute something like map[0] - 255 from the browser console, the loop would glitch out after a several loops.