Syphon / Syphon-Framework

Syphon is a Mac OS X technology to allow applications to share video and still images with one another in realtime, instantly.
Other
460 stars 81 forks source link

Understanding where my code goes wrong #66

Closed jcelerier closed 2 years ago

jcelerier commented 2 years ago

Hi,

When trying to read back textures from Syphon in my code, I only get the first line of the texture being sent. But I don't understand when things go wrong in the following (example, for me to try to understand) code:

void openServer()
{
  NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];

  SyphonServerDirectory *ssd = [SyphonServerDirectory sharedDirectory];
  NSArray *servers = [ssd serversMatchingName:NULL appName:NULL];
  if (servers.count != 0)
  {
    NSDictionary *desc = servers[0];
    m_receiver = [[SYPHON_CLIENT_UNIQUE_CLASS_NAME alloc]
        initWithServerDescription:desc
        context:my_gl_context
        options:NULL
        newFrameHandler:NULL
    ];
  }
  [pool drain];
}

void loadFrame()
{
  NSAutoreleasePool* p = [[NSAutoreleasePool alloc] init];
  auto img = [m_receiver newFrameImage];
  if(!img)
    return;

  makeContextCurrent();

  glBindTexture(GL_TEXTURE_RECTANGLE, img.textureName);
  {
    int w, h, f, r, g, b, a;
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_WIDTH, &w);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_HEIGHT, &h);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_INTERNAL_FORMAT, &f);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_RED_SIZE, &r);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_GREEN_SIZE, &g);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_BLUE_SIZE, &b);
    glGetTexLevelParameteriv(GL_TEXTURE_RECTANGLE, 0, GL_TEXTURE_ALPHA_SIZE, &a);
    qDebug() << "Gotten: " << w << h << (f == GL_RGBA8) << r << g << b << a;
    // Prints me 640 480 true 8 8 8 8 
  }

  NSSize sz = img.textureSize;
  m_pixels.resize(sz.width * sz.height * 4);
  glGetTexImage(GL_TEXTURE_RECTANGLE, 0, GL_RGBA, GL_UNSIGNED_BYTE, m_pixels.data());

  [img release];
  [p drain];
}

The glGetTexImage only writes the first line (if I fill my m_pixels array with some values, all the lines besides the first line are unchanged after the call). I tried using GL_TEXTURE_RECTANGLE_ARB, using GL_UNSIGNED_INT_8_8_8_8 or GL_UNSIGNED_INT_8_8_8_8_REV to retrieve the data instead, but the result is always the same (in the relevant data types). I also tried core and legacy/compat contexts, 4.1, 3.2 ... not much change.

vade commented 2 years ago

if memory serves, you need a GPU sync point - glFlushRenderersApple() or some such

bangnoise commented 2 years ago

glGetTexImage() has never worked for an IOSurface-backed GL texture - it's a poorly documented limitation of the OpenGL/IOSurface integration. I'm afraid you'll have to draw into an FBO then do a PBO download from the texture backing the FBO.

vade commented 2 years ago

Aah yes. Thanks for the reminder! I forgot about that!

jcelerier commented 2 years ago

ouch :) okay.. my original issue is that the rendering framework I'm using does not support importing textures in the GL_TEXTURE_RECTANGLE format, only GL_TEXTURE_2D... would changing this in my local Syphon code break compatibility with normal servers ?

vade commented 2 years ago

IIRC I dont believe IOSurface supports GL_TEXTURE_2D at all. I dont think that will work. You will need to blit to a GL_TEXTURE_2D backed FBO and use that texture.

jcelerier commented 2 years ago

okay, I'll try to fix the framework I'm using instead aha

jcelerier commented 2 years ago

thanks for the quick answers !