search

SceneView / sceneview-android

SceneView is a 3D and AR Android Composable and View with Google Filament and ARCore. This is a Sceneform replacement in Kotlin
Apache License 2.0
850 stars 161 forks source link

Unable to get a model to show in Java #64

Closed ninjz closed 2 years ago

ninjz commented 2 years ago

I'm trying to get the sample model displayed in my Java code. I've got everything working in SceneForm but I'm trying to migrate to this project for the memory management improvements. What am I doing wrong here? (This is a native android component I'm bridging for use in React Native so this class is being rendered inside of a Fragment.)

Here is my code:

class RNTExerciseView extends FrameLayout {

    private SceneView sceneView;
    private ModelNode characterModel;
    private Node cameraOrbit;
    private Camera camera;

    private ObjectAnimator currentlyPlaying;

    public RNTExerciseView(@NonNull Context context, ReactContext reactContext, int reactNativeViewId) {
        super(context);

        sceneView = new SceneView(context);
        sceneView.setBackgroundColor(Color.rgb(255, 255, 255));
        sceneView.setTransparent(true);
        sceneView.setZOrderOnTop(false);
        this.addView(sceneView);

        characterModel = new ModelNode();
        sceneView.addChild(characterModel);
        characterModel.setWorldPosition(new Float3(0, 0, -5));

        characterModel.loadModelAsync(
            context,
            "https://sceneview.github.io/assets/models/MaterialSuite.glb",
            null,
            true,
            true,
            new Float3(0, 0, 0), 
            null, 
            instance -> {
                return null;
            });
    }
}
ThomasGorisse commented 2 years ago

What error do you actually have? Can you copy/paste your logcat? At first look on your code, I think that your issue could come from the missing coroutine scope at the loadModelAsync call.

ThomasGorisse commented 2 years ago

You can still use the old (not archived but maintained) Sceneform system to load your Renderable from Java and outside of an Activity within SceneView. Evertything that you have available within Sceneform is also available within SceneView

ninjz commented 2 years ago

Thanks for taking a look at this. I've created a test project to show you a minimal case (don't mind that I accidentally named it TestSceneForm it is in fact using SceneView): https://drive.google.com/file/d/1i3RKFqr-TmjyA_CsCjeWSvQ0ihDvzKoi/view?usp=sharing

If you were to try to show the MaterialSuite.glb you can see a faint outline of the object: Screenshot 2022-04-14 at 11 56 03 AM

However, if you enable the backgroundColor by setting: sceneView.setBackgroundColor(Color.rgb(255, 255, 255)); the model is not shown.

For the other two models I've commented out, I'm unable to show them in either case with/without the background color set. There's no errors or anything in the logcat logs:

2022-04-14 12:02:37.360 21565-21565/? W/e.testscenefor: Redefining intrinsic method boolean java.lang.Thread.interrupted(). This may cause the unexpected use of the original definition of boolean java.lang.Thread.interrupted()in methods that have already been compiled.
2022-04-14 12:02:37.361 21565-21565/com.example.testsceneform D/CompatibilityChangeReporter: Compat change id reported: 171979766; UID 10148; state: ENABLED
2022-04-14 12:02:37.315 21565-21565/? I/e.testscenefor: Late-enabling -Xcheck:jni
2022-04-14 12:02:37.349 21565-21565/? W/re-initialized>: type=1400 audit(0.0:881): avc: granted { execute } for path="/data/data/com.example.testsceneform/code_cache/startup_agents/50cdcce7-agent.so" dev="dm-29" ino=139526 scontext=u:r:untrusted_app:s0:c148,c256,c512,c768 tcontext=u:object_r:app_data_file:s0:c148,c256,c512,c768 tclass=file app=com.example.testsceneform
2022-04-14 12:02:37.353 21565-21565/? V/studio.deploy: Startup agent attached to VM
2022-04-14 12:02:37.354 21565-21565/? V/studio.deploy: No existing instrumentation found. Loading instrumentation from instruments-88564b4d.jar
2022-04-14 12:02:37.355 21565-21565/? W/e.testscenefor: DexFile /data/data/com.example.testsceneform/code_cache/.studio/instruments-88564b4d.jar is in boot class path but is not in a known location
2022-04-14 12:02:37.356 21565-21565/? V/studio.deploy: Applying transforms with cached classes
2022-04-14 12:02:37.359 21565-21565/? W/e.testscenefor: Redefining intrinsic method java.lang.Thread java.lang.Thread.currentThread(). This may cause the unexpected use of the original definition of java.lang.Thread java.lang.Thread.currentThread()in methods that have already been compiled.
2022-04-14 12:02:37.485 21565-21565/com.example.testsceneform V/GraphicsEnvironment: ANGLE Developer option for 'com.example.testsceneform' set to: 'default'
2022-04-14 12:02:37.485 21565-21565/com.example.testsceneform V/GraphicsEnvironment: ANGLE GameManagerService for com.example.testsceneform: false
2022-04-14 12:02:37.485 21565-21565/com.example.testsceneform V/GraphicsEnvironment: Neither updatable production driver nor prerelease driver is supported.
2022-04-14 12:02:37.487 21565-21565/com.example.testsceneform D/NetworkSecurityConfig: No Network Security Config specified, using platform default
2022-04-14 12:02:37.487 21565-21565/com.example.testsceneform D/NetworkSecurityConfig: No Network Security Config specified, using platform default
2022-04-14 12:02:37.494 21565-21587/com.example.testsceneform D/libEGL: loaded /vendor/lib64/egl/libEGL_emulation.so
2022-04-14 12:02:37.494 21565-21587/com.example.testsceneform D/libEGL: loaded /vendor/lib64/egl/libGLESv1_CM_emulation.so
2022-04-14 12:02:37.495 21565-21587/com.example.testsceneform D/libEGL: loaded /vendor/lib64/egl/libGLESv2_emulation.so
2022-04-14 12:02:37.523 21565-21565/com.example.testsceneform W/e.testscenefor: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (unsupported, reflection, allowed)
2022-04-14 12:02:37.524 21565-21565/com.example.testsceneform W/e.testscenefor: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (unsupported, reflection, allowed)
2022-04-14 12:02:37.534 21565-21565/com.example.testsceneform D/HostConnection: createUnique: call
2022-04-14 12:02:37.534 21565-21565/com.example.testsceneform D/HostConnection: HostConnection::get() New Host Connection established 0xb400007989309650, tid 21565
2022-04-14 12:02:37.536 21565-21565/com.example.testsceneform D/HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_host_composition_v1 ANDROID_EMU_host_composition_v2 ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0 
2022-04-14 12:02:37.538 21565-21565/com.example.testsceneform D/EGL_emulation: eglCreateContext: 0xb400007989309f50: maj 3 min 0 rcv 3
2022-04-14 12:02:37.539 21565-21565/com.example.testsceneform D/EGL_emulation: eglMakeCurrent: 0xb400007989309f50: ver 3 0 (tinfo 0x7ba57ff080) (first time)
2022-04-14 12:02:37.542 21565-21565/com.example.testsceneform I/Filament: FEngine (64 bits) created at 0xb4000078752f3f00 (threading is enabled)
2022-04-14 12:02:37.542 21565-21590/com.example.testsceneform D/Filament: Using ASurfaceTexture
2022-04-14 12:02:37.542 21565-21590/com.example.testsceneform D/Filament: FEngine resolved backend: OpenGL
2022-04-14 12:02:37.543 21565-21590/com.example.testsceneform D/HostConnection: createUnique: call
2022-04-14 12:02:37.543 21565-21590/com.example.testsceneform D/HostConnection: HostConnection::get() New Host Connection established 0xb40000798930d850, tid 21590
2022-04-14 12:02:37.544 21565-21590/com.example.testsceneform D/HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_host_composition_v1 ANDROID_EMU_host_composition_v2 ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0 
2022-04-14 12:02:37.545 21565-21590/com.example.testsceneform D/EGL_emulation: eglCreateContext: 0xb40000798930d550: maj 3 min 0 rcv 3
2022-04-14 12:02:37.545 21565-21590/com.example.testsceneform D/EGL_emulation: eglMakeCurrent: 0xb40000798930d550: ver 3 0 (tinfo 0x7ba57ff100) (first time)
2022-04-14 12:02:37.546 21565-21590/com.example.testsceneform V/Filament: [Google (Apple)], [Android Emulator OpenGL ES Translator (Apple M1 Max)], [OpenGL ES 3.0 (4.1 Metal - 76.3)], [OpenGL ES GLSL ES 3.00]
2022-04-14 12:02:37.546 21565-21590/com.example.testsceneform V/Filament: Active workarounds: 
2022-04-14 12:02:37.595 21565-21585/com.example.testsceneform D/HostConnection: createUnique: call
2022-04-14 12:02:37.596 21565-21585/com.example.testsceneform D/HostConnection: HostConnection::get() New Host Connection established 0xb400007989306350, tid 21585
2022-04-14 12:02:37.601 21565-21585/com.example.testsceneform D/HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_host_composition_v1 ANDROID_EMU_host_composition_v2 ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0 
2022-04-14 12:02:37.601 21565-21585/com.example.testsceneform W/OpenGLRenderer: Failed to choose config with EGL_SWAP_BEHAVIOR_PRESERVED, retrying without...
2022-04-14 12:02:37.601 21565-21585/com.example.testsceneform W/OpenGLRenderer: Failed to initialize 101010-2 format, error = EGL_SUCCESS
2022-04-14 12:02:37.602 21565-21585/com.example.testsceneform D/EGL_emulation: eglCreateContext: 0xb40000798930c410: maj 3 min 0 rcv 3
2022-04-14 12:02:37.602 21565-21585/com.example.testsceneform D/EGL_emulation: eglMakeCurrent: 0xb40000798930c410: ver 3 0 (tinfo 0x7ba57ff180) (first time)
2022-04-14 12:02:37.606 21565-21585/com.example.testsceneform I/Gralloc4: mapper 4.x is not supported
2022-04-14 12:02:37.606 21565-21585/com.example.testsceneform D/HostConnection: createUnique: call
2022-04-14 12:02:37.606 21565-21585/com.example.testsceneform D/HostConnection: HostConnection::get() New Host Connection established 0xb40000798930c110, tid 21585
2022-04-14 12:02:37.606 21565-21585/com.example.testsceneform D/goldfish-address-space: allocate: Ask for block of size 0x100
2022-04-14 12:02:37.606 21565-21585/com.example.testsceneform D/goldfish-address-space: allocate: ioctl allocate returned offset 0x1efff8000 size 0x8000
2022-04-14 12:02:37.607 21565-21585/com.example.testsceneform W/Gralloc4: allocator 4.x is not supported
2022-04-14 12:02:37.610 21565-21585/com.example.testsceneform D/HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_host_composition_v1 ANDROID_EMU_host_composition_v2 ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0 
2022-04-14 12:02:37.737 21565-21592/com.example.testsceneform D/HostConnection: createUnique: call
2022-04-14 12:02:37.737 21565-21592/com.example.testsceneform D/HostConnection: HostConnection::get() New Host Connection established 0xb40000798930f890, tid 21592
2022-04-14 12:02:37.738 21565-21592/com.example.testsceneform D/HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_host_composition_v1 ANDROID_EMU_host_composition_v2 ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0 
2022-04-14 12:02:38.755 21565-21590/com.example.testsceneform D/EGL_emulation: app_time_stats: avg=63.26ms min=2.91ms max=831.46ms count=14
2022-04-14 12:02:39.329 21565-21577/com.example.testsceneform W/System: A resource failed to call close. 
2022-04-14 12:02:39.756 21565-21590/com.example.testsceneform D/EGL_emulation: app_time_stats: avg=3.37ms min=2.73ms max=5.25ms count=60
2022-04-14 12:02:40.760 21565-21590/com.example.testsceneform D/EGL_emulation: app_time_stats: avg=3.36ms min=2.80ms max=8.81ms count=59
2022-04-14 12:02:41.771 21565-21590/com.example.testsceneform D/EGL_emulation: app_time_stats: avg=3.99ms min=2.76ms max=15.95ms count=61

Maybe this is a lighting issue, but I'm not quite sure how to set the lights. When I was using SceneForm everything displayed from the start and I was satisfied with the default lighting of the scene.

ninjz commented 2 years ago

@ThomasGorisse Do you have any advice on how to proceed here? Thank you.

ThomasGorisse commented 2 years ago

Can you try the latestest version (0.5.1) ? We have fixed some parts on lightning.

ninjz commented 2 years ago

@ThomasGorisse I just tried the latest version (0.5.1) with the test project above and I'm getting the same result.

ninjz commented 2 years ago

@ThomasGorisse Do you think this is a problem related to me using the package in Java? If so, I'm wondering if it's worth trying to use Kotlin for this.

ThomasGorisse commented 2 years ago

You must provide your lifecycle on the loadModel() function. Must be something we forgot to change but it should not be nullable anymore to insure memory releasing.

ninjz commented 2 years ago

I've provided the activity's lifecycle in the loadModel() function:

    public CompletableFuture<Void> loadModels(Context context) {
        return ModelRenderable
                .builder()
                .setSource(context, Uri.parse("https://sceneview.github.io/assets/models/MaterialSuite.glb"))
                .setIsFilamentGltf(true)
                .setAsyncLoadEnabled(true)
                .build(this.getLifecycle()) // Provide lifecycle
                .thenAccept((modelRenderable) -> {
                    characterModel = new ModelNode();
                    sceneView.addChild(characterModel);

                    characterModel.setModel(modelRenderable);
                    characterModel.setPosition(new Float3(0f, 0f, -5f));
                    characterModel.setScale(new Float3(1f, 1f, 1f));
                });

Still no dice, getting the same result as before. Am I doing this incorrectly?

ninjz commented 2 years ago

I just ran the sample sample-model-viewer and got the same result when I removed the environment from the sceneView. I guess this is not an issue specific to the Java environment but more so that I'm just missing an environment.

Sceneform I guess provided a default environment so that everything works out of the box. How I could instantiate the same default environment so that I can get the same defaults as Sceneform? If I apply the environment in the sample when using my model, the lighting on the model looks very off.

ninjz commented 2 years ago

Update: I looked through the Sceneform project and saw that there's a default environment that is used during the initialization of the SceneView class: https://github.com/SceneView/sceneform-android/blob/92b8c8baee38b9acd496e640e9504b301acfae6b/core/src/main/java/com/google/ar/sceneform/SceneView.java#L410

I extracted the default_environment_ibl.ktx file and used that as the environment and everything looks as expected! Don't really know how to configure my own environments if there's any resources you could point me to that would be great.

Might be worth adding this as a default for the project so others like me who are migrating from Sceneform might not be as confused.

myselfuser1 commented 11 months ago

Are you running this on android emulator or real device?