ViewRenderable Material OpenSource 1.16.0

[bobekos]

I am really happy about the step to make Sceneform open source. The abandonment of SFBs and complete relying on filament was right.

But with version 1.16.0 it is not possible to render the viewrenderable. I assume that it's still the SFB object you use to put the ExternalTexture on it.

Can you provide us with the material of the object? Then I can simply use a GLTF object. It would be best to know what the original SFB model looked like.

[tpsiaki]

Hi @bobekos

I'm surprised that's not working, but you're right that using a gltf is probably preferable.

Here's a filament material that should work for you for view renderables:

material {
    "name" : "View",
    "defines" : [
        "baseColor"
    ],
    "parameters" : [
        {
           "type" : "samplerExternal",
           "name" : "viewTexture"
        },
        {
            "type" : "float2",
            "name" : "offsetUv"
        }
    ],
    "requires" : [
        "position",
        "uv0"
    ],
    "shadingModel" : "unlit",
    "blending" : "transparent",
    "doubleSided" : true
}

fragment {
    void material(inout MaterialInputs material) {
        prepareMaterial(material);

        vec2 uv = getUV0();

        if (!gl_FrontFacing) {
          uv.x = 1.0 - uv.x;
        }

        // Set offsetUv if we want to invert around an axis.
        // In front facing camera, set offsetUv.x to 1 and offsetUv.y to 0.
        uv.x = uv.x + materialParams.offsetUv.x * (1.0 - 2.0 * uv.x);
        uv.y = uv.y + materialParams.offsetUv.y * (1.0 - 2.0 * uv.y);

        material.baseColor = texture(materialParams_viewTexture, uv);
        material.baseColor.rgb = inverseTonemapSRGB(material.baseColor.rgb);
    }
}

You'll have to compile it using matc from Filament, but then you should be able to load that into Filament (or into the View Renderable directly).

Fwiw the view renderable sfb was built from an obj that looks like this (you can convert it to gltf, or if you inspect you'll see it's just a simple quad with what you'd expect from the UV set and normals):

o Plane
v -0.500000 0.000000 0.000000
v 0.500000 0.000000 0.000000
v -0.500000 1.000000 0.000000
v 0.500000 1.000000 0.000000
vt 0.0000 0.0000
vt 1.0000 0.0000
vt 1.0000 1.0000
vt 0.0000 1.0000
vn 0.0000 0.0000 1.0000
s off
f 1/1/1 2/2/1 4/3/1 3/4/1

[bobekos]

Hi @tpsiaki perfect, thank you very much.

[isnalla]

Hello @bobekos, could you share how you managed to solve the problem? I'm not sure how to use the material above to temporarily fix the issue until they push an update.

But with version 1.16.0 it is not possible to render the viewrenderable.

Do you mean it crashes the app? I'm getting Fatal signal native error when trying to build a ViewRenderable on 1.16.0

[bobekos]

@isnalla Yeah i got the same issue.

@tpsiaki Ok I thought the bug was related to the SFB, but it seems to be a problem in a method of class stream. This method is throwing a native exception:

@NonNull
        public Stream build(@NonNull Engine engine) {
            long nativeStream = nBuilderBuild(mNativeBuilder, engine.getNativeObject());
            if (nativeStream == 0) throw new IllegalStateException("Couldn't create Stream");
            return new Stream(nativeStream, engine);
        }

This is the exception:

2020-03-25 18:51:45.181 8378-8378/com.bobekos.gltf.debug A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 8378 (gltf.debug), pid 8378 (gltf.debug)
2020-03-25 18:51:45.295 8596-8596/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2020-03-25 18:51:45.295 8596-8596/? A/DEBUG: Build fingerprint: 'google/sargo/sargo:10/QQ2A.200305.002/6138846:user/release-keys'
2020-03-25 18:51:45.295 8596-8596/? A/DEBUG: Revision: 'MP1.0'
2020-03-25 18:51:45.295 8596-8596/? A/DEBUG: ABI: 'arm64'
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: Timestamp: 2020-03-25 18:51:45+0100
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: pid: 8378, tid: 8378, name: gltf.debug  >>> com.bobekos.gltf.debug <<<
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: uid: 10361
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: Cause: null pointer dereference
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x0  0000000000000000  x1  0000000000002f5a  x2  000000773e0e64e0  x3  0000000000000008
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x4  0000000000000000  x5  000000781dd2f5c8  x6  000000781dd74cb0  x7  000000781e063020
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x8  000000772063f2f0  x9  0000007716662340  x10 0000000000000000  x11 0000000000000000
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x12 000000781dd2e0c8  x13 0000000000000001  x14 000000772063f1e8  x15 0000000000000000
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x16 000000781a4568f0  x17 000000781a448860  x18 000000781e7b2000  x19 0000000000002f5a
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x20 000000772d78d6c0  x21 0000007fe73bfaf8  x22 000000000000c0fc  x23 0000000000000000
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x24 0000000000000010  x25 000000781e063020  x26 000000781dd74cb0  x27 0000000000000004
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     x28 0000007fe73bfba0  x29 0000007fe73bfba0
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG:     sp  0000007fe73bfa90  lr  000000772055949c  pc  0000007720552108
2020-03-25 18:51:45.296 8596-8596/? A/DEBUG: backtrace:
2020-03-25 18:51:45.297 8596-8596/? A/DEBUG:       #00 pc 0000000000092108  /data/app/com.bobekos.gltf.debug-OnvRmB7aEAdn7ZEbHPTgow==/base.apk!libgltfio-jni.so (offset 0xe7000) (BuildId: 3bef482b0a6dbb908e82f419ec05698e11ec23bc)

Any advice?

[prideout]

This is strange because Stream is implemented in libfilament, but that null pointer exception looks like it is being thrown from libgltfio (which doesn't touch streams).

Can you do me a favor and verify the exact Java codeline that's being invoked when the null pointer dereference is thrown from libgltfio-jni?

[bobekos]

@prideout Hi sure, Line 149 in Streams: long nativeStream = nBuilderBuild(mNativeBuilder, engine.getNativeObject());

The mNativeBuilder is initialized and "getNativeObject()" function return also a result.

I mean the stream builder is working when the camera stream is initialized. There is also the external texture which get builded. So maybe it is because of the other constructor of the external texture wich get called when you create a viewrenderable.

public ExternalTexture() {
    // Create the Android surface texture.
    SurfaceTexture surfaceTexture = new SurfaceTexture(0);
    surfaceTexture.detachFromGLContext();
    this.surfaceTexture = surfaceTexture;

    // Create the Android surface.
    surface = new Surface(surfaceTexture);

    // Create the filament stream.
    Stream stream =
        new Stream.Builder()
            .stream(surfaceTexture).build(EngineInstance.getEngine().getFilamentEngine());

    initialize(stream);
  }

Ps. Tested with the newest master

[tpsiaki]

@bobekos following up here - we have a repro of this locally and are looking into it.

[ArcherN9]

Thanks for documenting this @bobekos. @tpsiaki @prideout Is there a workaround we can look at in the meanwhile?

[prideout]

Unfortunately I think you'll need to wait until the next version of Filament is integrated into Sceneform.

[programmer35]

Any update on the fix for this? I am actively waiting on this to start using 1.16. I also ran into the same problem where all my viewRenderable.build().setView( etc... broke because of the long nativeStream = nBuilderBuild(mNativeBuilder, engine.getNativeObject()); of code causing a Fatal Error that doesn't happen in previous versions.