Multisample Color Attachments

[yetyman]

**I have a question*** I implemented custom color/depth attachments in 3.0.5 and to get those I essentially create my own framebuffer, bind the canvas's framebuffer's COLOR_ATTACHMENT_0 and then add two more attachments of my own.

I'm excited to use the MSAA built in to 4.0.5 but I'm a bit confounded on how to get it to work in the same way. I am rendering millions of quads with glDrawElementsInstanced(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0, availableAndRenderableCnt);

Here is how I was copying the color attachment from the built in frame buffer for my own framebuffer before MSAA

private void resizeColorAttachments(boolean init) {

    //bind the color texture to both the built-in framebuffer and my framebuffer

    //share the built in framebuffer's color buffer
    glBindFramebuffer(GL_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);

    FRAMEBUFFER_COLOR_TEX_INDEX = glGetFramebufferAttachmentParameteri(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
    int type1 = glGetFramebufferAttachmentParameteri(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE);

    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    if (type1 != GL_NONE) {
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, FRAMEBUFFER_COLOR_TEX_INDEX, 0);
    }
    CheckGPUErrors("Error binding built in framebuffer's attachments to custom framebuffer:");

    //make my own depth buffer
    glBindTexture(GL_TEXTURE_2D, MY_DEPTH_TEX_INDEX);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT32F, (int)viewportSize.getX(), (int)viewportSize.getY(), 0, GL_DEPTH_COMPONENT, GL_FLOAT, (IntBuffer)null);
    if (init)
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, MY_DEPTH_TEX_INDEX, 0);
    CheckGPUErrors("Error generating depth texture:");

    //make my own buffer of point indexes
    glBindTexture(GL_TEXTURE_2D, FRAMEBUFFER_POINT_INDEX_TEX_INDEX);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_R32UI, (int)viewportSize.getX(), (int)viewportSize.getY(), 0, INDEX_RASTER_FORMAT, INDEX_RASTER_TYPE, (IntBuffer)null);
    if (init)
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, FRAMEBUFFER_POINT_INDEX_TEX_INDEX, 0);
    CheckGPUErrors("Error generating pointIndex texture:");

    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    glBindTexture(GL_TEXTURE_2D, 0);
}

And here is how I have attempted to migrate it to handle multisampling.

private void resizeColorAttachments(boolean init) {

    //bind the color texture to both the built-in framebuffer and my framebuffer

    //share the built in framebuffer's color buffer
    glBindFramebuffer(GL_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);

    FRAMEBUFFER_COLOR_TEX_INDEX = glGetFramebufferAttachmentParameteri(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
    int type1 = glGetFramebufferAttachmentParameteri(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE);

    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    if (type1 != GL_NONE) {
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, FRAMEBUFFER_COLOR_TEX_INDEX, 0);
    }
    CheckGPUErrors("Error binding built in framebuffer's attachments to custom framebuffer:");

    //make my own depth buffer
    glBindTexture(GL_TEXTURE_2D, MY_DEPTH_TEX_INDEX);
    glTexImage2DMultisample(GL_TEXTURE_2D, 4, GL_DEPTH_COMPONENT32F, (int)viewportSize.getX(), (int)viewportSize.getY(), true);
    if (init)
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D_MULTISAMPLE, MY_DEPTH_TEX_INDEX, 0);
    CheckGPUErrors("Error generating depth texture:");

    //make my own buffer of point indexes
    glBindTexture(GL_TEXTURE_2D, FRAMEBUFFER_POINT_INDEX_TEX_INDEX);
    glTexImage2DMultisample(GL_TEXTURE_2D, 4, GL_R32UI, (int)viewportSize.getX(), (int)viewportSize.getY(), true);
    if (init)
        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D_MULTISAMPLE, FRAMEBUFFER_POINT_INDEX_TEX_INDEX, 0);
    CheckGPUErrors("Error generating pointIndex texture:");

    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    glBindTexture(GL_TEXTURE_2D, 0);
}

But I am getting errors here.

glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
if (type1 != GL_NONE) {
    glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, FRAMEBUFFER_COLOR_TEX_INDEX, 0);
}
CheckGPUErrors("Error binding built in framebuffer's attachments to custom framebuffer:");

The first issue and the one I am having the most issue deciphering is 1282 INVALID_OPERATION on the line I have bolded above. I assume the issue is that my assignment of the framebuffertexture is innapropriate for a multisample color attachment, but I do not know what about it is wrong. Is openGLFX using something other than a multisample color attachment on the built in frame buffer? Or am I simply failing to understand how to interact with a multisample color attachment? I am somewhat new to OpenGL, mostly to Multisampling.

[yetyman]

Okay. I now see that the COLOR_ATTACHMENT_0 on the built in framebuffer is no long a GL_TEXTURE and is now GL_RENDERBUFFER.

Sorry that I am quite new to OpenGL, if the structure of the built in frame buffer has changed, is there now a more efficient approach for me to add depth and color attachments? or is there a clean way to affect the logic I already had for multisampling?

[husker-dev]

Yes, openglfx uses GL_RENDERBUFFER instead of GL_TEXTURE to enable msaa. It can be found here.

But why would you add or override the built-in FBO attachments? It was intended as a simple "output" for a rendered content.

Tell me more about your case, maybe I can help.

[yetyman]

Thanks! I appreciate it. I am also open to easier solutions for what I am attempting to achieve. My goal is super fast presentation of a very large point cloud.

I am rendering and navigating massive points clouds embedded into a region in our app. To do this I am batching point data updates to the gfx card interspersed with render calls to avoid affecting performance.

I am adding a color attachment to be able to identify points by their index in our cloud based on user clicks. The second color attachment stored the point's index.

private Integer lookForPointInRasterizedIndexBuffer() {
    int x = (int)Math.min(Math.max(0, (int)pointRequested.getX() - SAMPLE_SIZE / 2), viewportSize.getX() - SAMPLE_SIZE);
    int y = (int)Math.min(Math.max(0, (int)pointRequested.getY() - SAMPLE_SIZE / 2), viewportSize.getY() - SAMPLE_SIZE);
    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    glReadBuffer(GL_COLOR_ATTACHMENT1);
    CheckGPUErrors("Error binding index buffer:");

    log.debug("blitting " + SAMPLE_SIZE + "x" + SAMPLE_SIZE + " at " + x + "," + y);

    int[] pixels = new int[SAMPLE_SIZE * SAMPLE_SIZE];
    glReadPixels(x, y, SAMPLE_SIZE, SAMPLE_SIZE, INDEX_RASTER_FORMAT, INDEX_RASTER_TYPE, pixels);
    CheckGPUErrors("Error reading pixel data:");

    Integer index = null;

    try{
        index = findNearestPoint(pixels, (int)pointRequested.getX() - x, (int)pointRequested.getY() - y);
    } catch (Exception ex){
        ReportHelper.overTime(ReportName.ClickPointSearch3D, 2000, ()->{
            log.info("clicked outside point bounds", ex);
        });
        index = null;
    }

    glBindFramebuffer(GL_FRAMEBUFFER, MY_FRAMEBUFFER_INDEX);
    glReadBuffer(GL_COLOR_ATTACHMENT0);
    glDrawBuffer(GL_COLOR_ATTACHMENT0);

    return index;
}

I am not affecting the built-in FBO's structure. I am binding its first color attachment to my own FBO, adding some layers to my FBO for extra data, and then rendering to my FBO to get all the data i need. Does that explain my intent?

Also thank you for the link. I may be able to cut my own depth buffer out of the mix now that the multisample fbo has one. I still need to be able to encode the point index data of whatever was rendered though, so i do still have a goal of adding a second color_attachment to store that information

[yetyman]

For context the only alternative I am aware of for knowing which point a user clicked is simulating my own cloud to perspective ray test on the cpu, mirroring all of the perspective logic. Which even with a good oct tree impl was still quite slow/error prone at scale.

[husker-dev]

From what I understand, you can make the following algorithm:

---

At the resizing event, attach GL_COLOR_ATTACHMENT1 to the built-in FBO, similar to the built-in code:

val myColorBuffer = glGenRenderbuffers()
glBindRenderbuffer(GL_RENDERBUFFER, myColorBuffer )
// Try with and without msaa, one of them should work
// glRenderbufferStorageMultisample(GL_RENDERBUFFER, msaa, GL_RGBA8, width, height)
// glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8, width, height)
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_RENDERBUFFER, myColorBuffer )

---

In rendering event:

1. Load points into VAO
2. Call:

   glDrawBuffers([GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1])

3. Draw these points into the built-in FBO, taking into account two attachments in the shader
4. Switch the current read buffer to GL_COLOR_ATTACHMENT1:

   glReadBuffer(GL_COLOR_ATTACHMENT1)

5. Get pixel value
6. Return custom values back:

   glDrawBuffers([GL_COLOR_ATTACHMENT0])
   glReadBuffer(GL_COLOR_ATTACHMENT0)

---

In theory this method should work

[yetyman]

This appears quite a lot simpler than what I have done, thank you I'll give it a try!

[yetyman]

At the resizing event, attach GL_COLOR_ATTACHMENT1 to the built-in FBO, similar to the built-in code: @husker-dev should i be able to generate a render buffer in the resize event? I cannot and get java crashes when I do so. I have been setting a flaw for resize invalidation in the resize event and recreating things for resize inside of the render event.

[husker-dev]

Can you show me the crash log?

[yetyman]

Its an access violation. I always assumed that it happened because the resize event wasn't in the correct context for affecting openGL. Should this not be happening?

This occurs after some delay whenever glGenRenderbuffers(); (or other gl calls) are made in the resize event for me

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffc9b69f62d, pid=84532, tid=37780
#
# JRE version: OpenJDK Runtime Environment Corretto-17.0.5.8.1 (17.0.5+8) (build 17.0.5+8-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-17.0.5.8.1 (17.0.5+8-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  0x00007ffc9b69f62d
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\myusername\IdeaProjects\myappsname\hs_err_pid84532.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/corretto/corretto-17/issues/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

[husker-dev]

Are you using 'async' option?

Also, can you show the stacktrace in C:\Users\myusername\IdeaProjects\myappsname\hs_err_pid84532.log?

[yetyman]

I have async off in new GLCanvas as i started to experience rendering artifacts. (i would get streaks of small squares that were updating slower than the rest of the canvas. they would go away in a moment)

hs_err_pid84532-sanitized.log

[husker-dev]

You should call gl functions only from GLCanvas events. There is an onResize event for these purposes. It is an OpenGL limitation, because it's single threaded

---

Are you experiencing artifacts on Windows? Can you tell me more? Do you specify any parameters for JavaFX?

[yetyman]

Oh, that is very much my bad, that is the Fn i have hooked into the canvas.addOnReshapeEvent(pipeline::reshape); event. I forgot that i was also calling it from JFX's resize due to some resizing issue i was having at one point.

[yetyman]

With that resolved, now I am getting Invalid Framebuffer Operation(1286) between each render call. the error is not present at the end of the render call and is present before any of my calls starting after the first frame. What could that be? How do I approach identifying it?

[yetyman]

oh. my fbo status is not GL_FRAMEBUFFER_COMPLETE it is 36182

[yetyman]

okay. Now I've resolved all of the crash/error issues; I'm just getting a blank screen and will need to debug. Thank you @husker-dev . Now to try the new RenderDoc support.

[yetyman]

You should call gl functions only from GLCanvas events. There is an onResize event for these purposes. It is an OpenGL limitation, because it's single threaded

Are you experiencing artifacts on Windows? Can you tell me more? Do you specify any parameters for JavaFX?

Yes a bit. But I think some of those could have been from using my own FBO. If I continue to experience them I will let you know.

[yetyman]

Alirghty! I have migrated to Render buffers and msaa rendering and its working!! It looks very nice now and my rendering artifacts are gone, exactly what i was hoping msaa could give me.

I can no longer read pixels from the second color attachment as it apparently needs to be down sampled first https://stackoverflow.com/a/803545/6307037

I'll let you know if i'm run into anymore roadblocks. Thank you very much!

[husker-dev]

No problem. I will be glad if you star a project :)

[yetyman]

Alright I have one more question. I am attempting to get the blitting working and I'm getting INVALID OPERATION on the blit. I've read the docs and imagine that the reason for the error is that i'm missing something for sampling the multisample render buffer but I don't know for sure.

Here's the sample code

creating the color attachment on the built in frame buffer in the resize code

//add color attachment to built-in framebuffer
POINT_INDEX_RENDER_BUFFER_INDEX = glGenRenderbuffers();
glBindRenderbuffer(GL_RENDERBUFFER, POINT_INDEX_RENDER_BUFFER_INDEX );

int msaa = 4;
glRenderbufferStorageMultisample(GL_RENDERBUFFER, msaa, GL_R32UI, w, h);

glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_RENDERBUFFER, POINT_INDEX_RENDER_BUFFER_INDEX);
CheckGPUErrors("Error creating and binding render buffer for attachment "+attachment+":");

creating an fbo with color attachment as needed of the same format as the added color attachment on the built in fbo. then blitting

//create tiny fbo as target for blit
if(MY_BLIT_FBO == null) {
    MY_BLIT_FBO = glGenFramebuffers();
    glBindFramebuffer(GL_FRAMEBUFFER, MY_BLIT_FBO);
    int myBlitRenderBuffer = glGenRenderbuffers();
    glBindRenderbuffer(GL_RENDERBUFFER, myBlitRenderBuffer );

    // must not multisample. multisample cannot be read.
    glRenderbufferStorage(GL_RENDERBUFFER, GL_R32UI, SAMPLE_SIZE, SAMPLE_SIZE);

    glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, myBlitRenderBuffer);

    int fboStatus = glCheckFramebufferStatus(GL_FRAMEBUFFER);
    if (fboStatus != GL_FRAMEBUFFER_COMPLETE) {
        log.error("FBO status: " + fboStatus);
    }
    glBindFramebuffer(GL_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);

}

int[] pixels = new int[SAMPLE_SIZE * SAMPLE_SIZE];

// Bind the multisampled FBO for reading
glBindFramebuffer(GL_READ_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);
glReadBuffer(GL_COLOR_ATTACHMENT1);
// Bind the blitFBO for drawing
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, MY_BLIT_FBO);//blitFBO has a color_attachment_0 of exactly sample size for this
glDrawBuffer(GL_COLOR_ATTACHMENT0);

CheckGPUErrors("Error binding buffers for blit:");

// Blit the multisampled FBO to the normal FBO
glBlitFramebuffer(x, y, SAMPLE_SIZE, SAMPLE_SIZE, 0, 0, SAMPLE_SIZE, SAMPLE_SIZE, GL_COLOR_BUFFER_BIT, GL_NEAREST);
CheckGPUErrors("Error blitting data:");

When i make the blit call I get Invalid Operation(1282) Are there extra gl calls i need to make to indicate that the read render buffer is multisampled?

[husker-dev]

Can you show the full exception or stack trace?

[yetyman]

That is the full exception? Is there a way to get more informative error messages from opengl??????

[yetyman]

i'm calling glGetError() to find that

[husker-dev]

Ok, I thought it was a java exception

[yetyman]

nope, OpenGL is failing on my blit call. I assume its some kind of format mismatch or multisample reading failure, but I can't figure out more

[yetyman]

Ah. I have identified that the issue was in the size of the destination fbo's color attachment. I had tried to be clever and create an fbo that was only the size of my sample area, but apparently that does not work

[husker-dev]

Have you solve the problem?

[yetyman]

Almost. I am getting all 0s after the blit now but no errors are thrown.

[yetyman]

I am trying to use the new RenderDoc.bind or RenderDoc.startFrameCapture options.

I have tried start/end surrounding individual render calls and surrounding init/dispose. In either case, I open my app. inject with renderdoc, create the new GLCanvas, and then see Renderdoc show it for a moment and then it deletes its own tab for the injected process giving me nothing....

How can i get Renderdoc attached?

[yetyman]

After some further investigation my blit/read pixels operation is randomly missing data. partial sets of the pixels i read are missing. I think this is about the gfx card's concurrency with sampling the FBO because the missing areas are sporadic, random, and block shaped. I imagine that if I attach render doc I will be able to understand more.

A note, sometimes that missing data is actually wrong data where closer quads should overlap the smaller quads in the bg, but the closer quad data isn't there in the read. The frame has already rendered and is correct in appearance so I'm confused about how the multisampled color_attachments could have bad z order data.

Is it safe to be reading the built in FBO's data from the last frame at the start of the render event before i call clear?

[husker-dev]

I haven't worked much with RenderDoc, so I won't be able to help much. Keep in mind that for it to work on Windows you need to change interopType to Blit in the constructor. Described here. Also, he is unlikely to show you a preview of the final image, because it doesn't know which FBO to show.

---

About z-index Add depth-renderbuffer to your temporary FBO that is used to blitting. It will solve the incorrect values. Example here.

---

About missing data (not your case I think). OpenGL has internal caching of operations - it can accumulate approximately 10 operations, and only at 11 it sends it to the video card (Not 10 actually, it is example). To manually send operations to the graphics card you need to call glFinish. Try to add this between some lines, and test where it actually should be.

---

About reading from FBO at beginning You can, but is not recomended.

[yetyman]

Thank you! That gives me a lot I can look into.

[yetyman]

In adding a depth-renderbuffer to my temporary FBO, do I also need to perform a second blit into my temporary fbo with the depth_buffer_bit set? I thought that the depth buffer was essentially an intermediary used before the fragment shader which should already have happened when sampling the FBO late/at the start of the next frame. So I had thought that the blit would be copying data that was already z ordered by the msaa depth buffer before the previous frame was drawn.

[husker-dev]

if you won't draw anything into texture after blitting, then the depth bit is unnecessary

[yetyman]

I have tried each of these suggestions and it is failing. I do have glFinish() between each of my calls. I realized at some point that I was not clearing the blit fbo each frame and I am now doing that. The sporadic data issue is gone and I simply never see data in the FBO. This leads me to believe that the blit or previous draw is slow and asynchronous compared to the pixel read i do next. such that when i was getting data, it was because i had clicked in a similar place previously.

Would that make sense? does my initial draw on the built in fbo or my blit into the temporary fbo need to have memory barriers between them to ensure that data is available? Am I able to use memory barriers with the current OpenGLFX api? What memory barriers or locks do i need to ensure that I can sequentially call built in fbo draw --> blit to blitting fbo --> read pixels of blitting FBO?

[yetyman]

or should i be reading the blit from a front buffer that has fully resolved a frame late?

[yetyman]

For context, here is the render call's draw + the blitting code unwrapped.

glDrawElementsInstanced(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0, availableAndRenderableCnt);
if (pointRequested != null) {

    int w = 21;//SAMPLE_SIZE
    int h = 21;//SAMPLE_SIZE

    int x = (int)Math.min(Math.max(0, (int)pointRequested.getX() - w / 2), viewportSize.getX() - w);
    int y = (int)Math.min(Math.max(0, (int)pointRequested.getY() - h / 2), viewportSize.getY() - h);
    CheckGPUErrors("Error binding index buffer:");

    log.debug("blitting " + w + "x" + h + " at " + x + "," + y);

    int[] pixels = new int[w * h];

    // Bind the blitFBO for drawing
    CheckGPUErrors("Error before blit:");
    if(MY_BLIT_FBO == null) {
        MY_BLIT_FBO = glGenFramebuffers();
        glBindFramebuffer(GL_FRAMEBUFFER, MY_BLIT_FBO);
        boolean useRenderBuffer = true;
        if(useRenderBuffer) {
            myBlitRenderBuffer = createAndBindRenderBuffer(myBlitRenderBuffer,
                    GL_R32UI, GL_COLOR_ATTACHMENT0, (int)viewportSize.getX(), (int)viewportSize.getY(), 0);

            myBlitDepthRenderBuffer = createAndBindRenderBuffer(myBlitDepthRenderBuffer,
                    GL_DEPTH_COMPONENT, GL_DEPTH_ATTACHMENT, (int)viewportSize.getX(), (int)viewportSize.getY(), 0);
        }
        int fboStatus = glCheckFramebufferStatus(GL_FRAMEBUFFER);
        if (fboStatus != GL_FRAMEBUFFER_COMPLETE) {
            log.error("FBO status: " + fboStatus);
        }
        glBindFramebuffer(GL_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);
        CheckGPUErrors("Error creating fbo and buffers for blit:");
    }
    glFinish();
    glBindFramebuffer(GL_FRAMEBUFFER, MY_BLIT_FBO);
    glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);

    glFinish();
    // Bind the multi-sampled FBO for reading
    glBindFramebuffer(GL_READ_FRAMEBUFFER, BUILT_IN_FRAMEBUFFER_INDEX);
    CheckGPUErrors("Error binding read framebuffer for blit:");
    glReadBuffer(GL_COLOR_ATTACHMENT1);
    CheckGPUErrors("Error binding read buffer for blit:");

    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, MY_BLIT_FBO);//blitFBO has a color_attachment_0 of exactly sample size for this
    glDrawBuffer(GL_COLOR_ATTACHMENT0);

    CheckGPUErrors("Error binding draw buffers for blit:");
    // Blit the multi-sampled FBO to the normal FBO
    glFinish();
    glBlitFramebuffer(x, y, w, h, x, y, w, h, GL_COLOR_BUFFER_BIT, GL_NEAREST);
    CheckGPUErrors("Error blitting data:");

    glFinish();
    //Bind the normal FBO for reading
    glBindFramebuffer(GL_READ_FRAMEBUFFER, MY_BLIT_FBO);
    CheckGPUErrors("Error blitting:");

    glBindFramebuffer(GL_READ_FRAMEBUFFER, 0);
    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);

    // Read the pixels!
    glBindFramebuffer(GL_FRAMEBUFFER, MY_BLIT_FBO);
    glFinish();
    glReadPixels(x, y, w, h, INDEX_RASTER_FORMAT, INDEX_RASTER_TYPE, pixels);
    glFinish();
    CheckGPUErrors("Error reading pixel data:");
}

and here is the unwrapped reshape event's code for adding the index data color attachment

int msaa = 4;
int w = (int)viewportSize.getX();
int h = (int)viewportSize.getY()

if(MY_DEPTH_BUFFER_INDEX == 0)
    MY_DEPTH_BUFFER_INDEX = glGenRenderbuffers();
glBindRenderbuffer(GL_RENDERBUFFER, MY_DEPTH_BUFFER_INDEX );

glRenderbufferStorageMultisample(GL_RENDERBUFFER, msaa, GL_DEPTH_COMPONENT, w, h);

glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, MY_DEPTH_BUFFER_INDEX);
CheckGPUErrors("Error creating and binding render buffer for attachment "+GL_DEPTH_ATTACHMENT+":");

if(MY_POINT_INDEX_RENDER_BUFFER_INDEX == 0)
    MY_POINT_INDEX_RENDER_BUFFER_INDEX = glGenRenderbuffers();
glBindRenderbuffer(GL_RENDERBUFFER, MY_POINT_INDEX_RENDER_BUFFER_INDEX );

// Try with and without msaa, one of them should work
glRenderbufferStorageMultisample(GL_RENDERBUFFER, msaa, GL_R32UI, w, h);

glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_RENDERBUFFER, MY_POINT_INDEX_RENDER_BUFFER_INDEX);
CheckGPUErrors("Error creating and binding render buffer for attachment "+GL_COLOR_ATTACHMENT1+":");

int fboStatus = glCheckFramebufferStatus(GL_FRAMEBUFFER);
if (fboStatus != GL_FRAMEBUFFER_COMPLETE) {
    log.error("FBO status: " + fboStatus);
}

[yetyman]

The issue is resolved. It turns out that my culprit was the blit command. This was my blit command. glBlitFramebuffer(x, y, w, h, x, x, w, h, bufferBit, interpolationMode);

and this is the blit command that works. I also adjusted my blit fbo to only need to be WxH size and the read pixels call to do its job at 0,0. glBlitFramebuffer(x, y, x+w, y+h, 0, 0, w, h, bufferBit, interpolationMode);

It totally went past me that the blit command needed the outer corners of the box rather than x,y + w,h.

Thank you for all your help @husker-dev . The issue is resolved!

[husker-dev]

Good luck with your development!