processing / processing-video

GStreamer-based video library for Processing
276 stars 129 forks source link

movieEvent bug in combination with mipmaps #218

Open clankill3r opened 1 year ago

clankill3r commented 1 year ago

I tried to isolate the problem, but if I get rid of the PGraphics for example the bug seems to disappear.

If I run the below script I get:

Screenshot 2023-07-05 at 13 57 00
import processing.video.*;

PGraphics big;
PGraphics small;

boolean from_big_pg = true;

Movie movie;

void setup() {
  size(320, 180, P2D);
  movie = new Movie(this, "big.mp4");
  movie.loop();

  big = createGraphics(1280, 720, P2D);
  small = createGraphics(32, 18, P2D);
}

//void movieEvent(Movie m) {
//  m.read();
//}

void draw() {

  if (movie.available()) {
    movie.read();
  }

  // only works with `void movieEvent(Movie m)`
  // and not with `if (movie.available())` (it stays grey)

  gl_texture(movie).usingMipmaps(true, 3);
  gl_texture(big).usingMipmaps(true, 3);
  gl_texture(small).usingMipmaps(true, 3);

  big.beginDraw();
  big.image(movie, 0, 0, big.width, big.height);
  big.endDraw();

  small.beginDraw();
  small.image(from_big_pg ? big : movie, 0, 0, small.width, small.height);
  small.endDraw();

  image(small, 0, 0, 320, 180);
}

public Texture gl_texture(PImage img) {
  return ((PGraphicsOpenGL)g).getTexture(img);
}

If I comment out the following: (which is not what I want):

gl_texture(movie).usingMipmaps(true, 3);
gl_texture(big).usingMipmaps(true, 3);
gl_texture(small).usingMipmaps(true, 3);

Then it works again:

Screenshot 2023-07-05 at 14 02 43

Now the interesting part:

If I use:

void movieEvent(Movie m) {
  m.read();
}

Instead of:

if (movie.available()) {
  movie.read();
}

Then the usingMipmaps part works...

(Also if the usingMipmaps is before if (movie.available()) { then there is a nullpointerException).

The video file: https://github.com/processing/processing-video/assets/738650/f7a6ba31-b697-49fb-aeac-110823d0453d

clankill3r commented 9 months ago

Ok, I figured something out.

    int LINEAR = 3;
    int BILINEAR = 4;
    int sampling = (frameCount & 1) == 0 ? BILINEAR : LINEAR;
    gl_texture(passes[3]).usingMipmaps(true, sampling);

Switching the filter for the mipmap every frame makes it works, of course this is a workaround and not a fix but I can live with it for now.