google / filament

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
https://google.github.io/filament/
Apache License 2.0
17.44k stars 1.84k forks source link

How can I pass a java HardwareBuffer to Texture on android platform? #7700

Closed JungleKong closed 4 months ago

JungleKong commented 4 months ago

I want to use a share texture provided by another process, represented by HardwareBuffer. When I try to pass it to Filament Texture, but setExternalImage takes the EGLImage as parameter.

So I tried to add an API to OpenGLDriver, it looks like that:

void OpenGLDriver::setExternalHardwareBuffer(Handle<HwTexture> th, void* hwbuffer) {
    GLTexture* t = handle_cast<GLTexture*>(th);
    assert(t);

    AcquiredImage acquiredImage = mPlatform.transformAcquiredImage({
            hwbuffer, nullptr, nullptr, nullptr });

    bindTexture(OpenGLContext::DUMMY_TEXTURE_BINDING, t);
    if (mPlatform.setExternalImage(acquiredImage.image, t->externalTexture)) {
        // the target and id can be reset each time
        t->gl.target = t->externalTexture->target;
        t->gl.id = t->externalTexture->id;
        t->gl.targetIndex = (uint8_t)OpenGLContext::getIndexForTextureTarget(t->gl.target);
        bindTexture(OpenGLContext::DUMMY_TEXTURE_BINDING, t);
    }
}

Then a Texture JNI interface will be OK as usual, an I call it

val texture = Texture.Builder()
                .width(1024)
                .height(1024)
                .sampler(Texture.Sampler.SAMPLER_EXTERNAL)
                .format(Texture.InternalFormat.RGBA8)
                .levels(1)
                .build(engine)
texture.setExternalHardwareBuffer(engine, hardwareBuffer)

I am certain that the hardwareBuffer can lock right value in native(OpenGLDriver::setExternalHardwareBuffer), it just dose not work. So, where did I go wrong, and how can I fix it?


By the way, if I gen Texture in Java directly and then bind texId with HardwareBuffer, it works very well. It looks like as bellow: At fisrt, I create Engine use Java EGL context, engine = Engine.create(sharedContext) and then gen Texture ID and prepare AHardwareBuffer, pass them to native cpp

EGLClientBuffer clientBuffer = glext::eglGetNativeClientBufferANDROID(buffer);
EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
EGLint eglImageAttributes[] = {EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE};
EGLImageKHR eglImage = glext::eglCreateImageKHR(display, EGL_NO_CONTEXT, EGL_NATIVE_BUFFER_ANDROID,
                                   clientBuffer, eglImageAttributes);
glBindTexture(GL_TEXTURE_2D, texId);
glext::glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, (GLeglImageOES) eglImage);

In Java, pass the texId to Filament Texture

val texture = Texture.Builder()
                .width(1024)
                .height(1024)
                .sampler(Texture.Sampler.SAMPLER_2D)
                .format(Texture.InternalFormat.RGBA8)
                .importTexture(texId)
                .levels(1)
                .build(engine)

Or how can I share Texture between two Filament processes?