SceneView / sceneform-android

Sceneform Maintained is an ARCore Android SDK with Google Filament as 3D engine. This is the continuation of the archived Sceneform
https://sceneview.github.io/sceneform-android/
Apache License 2.0
633 stars 146 forks source link

Model from sample project not loading in Android Studio emulator #304

Closed davidliang104 closed 1 year ago

davidliang104 commented 2 years ago

I am trying to run the AR model viewer from the sample projects folder here on Android Studio 4.2.2 with a Pixel 2 API 29 x86 emulator.

When running the app in the emulator, it works as expected until this part:

Run

After I tap the screen to spawn the model, a "Loading" message appears, but nothing happens. The following lines are constantly repeated in the logcat until the app is forcibly stopped:

2022-02-04 02:12:32.389 8238-8347/com.example.modelviewer E/ACameraMetadata: getConstEntry: cannot find metadata tag 65576
2022-02-04 02:12:32.393 8238-8238/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xd81e6c80: ver 3 1 (tinfo 0xe3801bd0)
2022-02-04 02:12:32.396 8238-8238/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-04 02:12:32.396 8238-8347/com.example.modelviewer E/ACameraMetadata: getConstEntry: cannot find metadata tag 65576
2022-02-04 02:12:32.396 8238-8238/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-04 02:12:32.416 8238-8238/com.example.modelviewer E/ACameraMetadata: getConstEntry: cannot find metadata tag 65576
2022-02-04 02:12:32.416 8238-8238/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xc0f9b800: ver 3 1 (tinfo 0xe3801bd0)
2022-02-04 02:12:32.417 8238-8238/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xd81e6c80: ver 3 1 (tinfo 0xe3801bd0)
2022-02-04 02:12:32.418 8238-8238/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xc0f9b800: ver 3 1 (tinfo 0xe3801bd0)
2022-02-04 02:12:32.419 8238-8238/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xd81e6c80: ver 3 1 (tinfo 0xe3801bd0)
2022-02-04 02:12:32.450 8238-8238/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-04 02:12:32.450 8238-8238/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-04 02:12:32.554 8238-8347/com.example.modelviewer E/native: E0204 02:12:32.554694    8347 camera_image_stream.cc:270] FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata.
    === Source Location Trace: ===
    third_party/arcore/ar/camera/android/metadata_cache.cc:52
    third_party/arcore/ar/camera/android/metadata_extraction.cc:53

    === Source Location Trace: ===
    third_party/arcore/ar/camera/android/camera_image_stream.cc:204

There was a similar issue faced by several users who reported it on the Google ARCore SDK issues page, but it doesn't look like it's been resolved. Is there anything I can do to fix this error?

In case this could help, here is the build.gradle file for the app:

plugins {
    id 'com.android.application'
}

android {
    compileSdkVersion 31
    buildToolsVersion "32.0.0"

    defaultConfig {
        applicationId "com.example.modelviewer"
        minSdkVersion 27
        targetSdkVersion 29
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

dependencies {

    implementation "com.gorisse.thomas.sceneform:sceneform:1.20.4"
    implementation 'androidx.appcompat:appcompat:1.4.1'
    implementation 'com.google.android.material:material:1.5.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.3'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

And these are the SDK packages installed

SDKs

I also installed Google Play Services for AR 1.29x86 from here onto the emulator device as recommended by this webpage when running ARCore.

ThomasGorisse commented 2 years ago

Are you using the sample as it is or did you modify anything? the glb, the ARCore session config,...

davidliang104 commented 2 years ago

I wasn't able to build the project straight after downloading the folder, so I created an empty project and copied the code. It's the same as far as I can tell.

public class MainActivity extends AppCompatActivity implements
        FragmentOnAttachListener,
        BaseArFragment.OnTapArPlaneListener,
        BaseArFragment.OnSessionConfigurationListener,
        ArFragment.OnViewCreatedListener{

    private ArFragment arFragment;
    private Renderable model;
    private ViewRenderable viewRenderable;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        getSupportFragmentManager().addFragmentOnAttachListener(this);

        if (savedInstanceState == null) {
            if (Sceneform.isSupported(this)) {
                getSupportFragmentManager().beginTransaction()
                        .add(R.id.arFragment, ArFragment.class, null)
                        .commit();
            }
        }

        loadModels();
    }

    @Override
    public void onAttachFragment(@NonNull FragmentManager fragmentManager, @NonNull Fragment fragment) {
        if (fragment.getId() == R.id.arFragment) {
            arFragment = (ArFragment) fragment;
            arFragment.setOnSessionConfigurationListener(this);
            arFragment.setOnViewCreatedListener(this);
            arFragment.setOnTapArPlaneListener(this);
        }
    }

    @Override
    public void onSessionConfiguration(Session session, Config config) {
        if (session.isDepthModeSupported(Config.DepthMode.AUTOMATIC)) {
            config.setDepthMode(Config.DepthMode.AUTOMATIC);
        }
    }

    @Override
    public void onViewCreated(ArSceneView arSceneView) {
        arFragment.setOnViewCreatedListener(null);

        // Fine adjust the maximum frame rate
        arSceneView.setFrameRateFactor(SceneView.FrameRate.FULL);
    }

    public void loadModels() {
        WeakReference<MainActivity> weakActivity = new WeakReference<>(this);
        ModelRenderable.builder()
                .setSource(this, Uri.parse("https://storage.googleapis.com/ar-answers-in-search-models/static/Tiger/model.glb"))
                .setIsFilamentGltf(true)
                .setAsyncLoadEnabled(true)
                .build()
                .thenAccept(model -> {
                    MainActivity activity = weakActivity.get();
                    if (activity != null) {
                        activity.model = model;
                    }
                })
                .exceptionally(throwable -> {
                    Toast.makeText(
                            this, "Unable to load model", Toast.LENGTH_LONG).show();
                    return null;
                });
        ViewRenderable.builder()
                .setView(this, R.layout.view_model_title)
                .build()
                .thenAccept(viewRenderable -> {
                    MainActivity activity = weakActivity.get();
                    if (activity != null) {
                        activity.viewRenderable = viewRenderable;
                    }
                })
                .exceptionally(throwable -> {
                    Toast.makeText(this, "Unable to load model", Toast.LENGTH_LONG).show();
                    return null;
                });
    }

    @Override
    public void onTapPlane(HitResult hitResult, Plane plane, MotionEvent motionEvent) {
        if (model == null || viewRenderable == null) {
            Toast.makeText(this, "Loading...", Toast.LENGTH_SHORT).show();
            return;
        }

        // Create the Anchor.
        Anchor anchor = hitResult.createAnchor();
        AnchorNode anchorNode = new AnchorNode(anchor);
        anchorNode.setParent(arFragment.getArSceneView().getScene());

        // Create the transformable model and add it to the anchor.
        TransformableNode model = new TransformableNode(arFragment.getTransformationSystem());
        model.setParent(anchorNode);
        model.setRenderable(this.model)
                .animate(true).start();
        model.select();

        Node titleNode = new Node();
        titleNode.setParent(model);
        titleNode.setEnabled(false);
        titleNode.setLocalPosition(new Vector3(0.0f, 1.0f, 0.0f));
        titleNode.setRenderable(viewRenderable);
        titleNode.setEnabled(true);
    }
}
ThomasGorisse commented 2 years ago

Try removing this:

    @Override
    public void onSessionConfiguration(Session session, Config config) {
        if (session.isDepthModeSupported(Config.DepthMode.AUTOMATIC)) {
            config.setDepthMode(Config.DepthMode.AUTOMATIC);
        }
    }
davidliang104 commented 2 years ago

After removing that method and the other two mentions of the session configuration listener, the same problems appear (or don't appear). The "Loading" text pops up sometimes, but no model comes up. The logcat still repeats these lines:

2022-02-05 21:18:40.099 8380-8380/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-05 21:18:40.099 8380-8380/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-05 21:18:40.153 8380-8495/com.example.modelviewer E/ACameraMetadata: getConstEntry: cannot find metadata tag 65576
2022-02-05 21:18:40.155 8380-8380/com.example.modelviewer E/ACameraMetadata: getConstEntry: cannot find metadata tag 65576
2022-02-05 21:18:40.156 8380-8380/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xe08e0c80: ver 3 1 (tinfo 0xe090cd40)
2022-02-05 21:18:40.157 8380-8380/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xd53cb0a0: ver 3 1 (tinfo 0xe090cd40)
2022-02-05 21:18:40.158 8380-8380/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xe08e0c80: ver 3 1 (tinfo 0xe090cd40)
2022-02-05 21:18:40.159 8380-8380/com.example.modelviewer D/EGL_emulation: eglMakeCurrent: 0xd53cb0a0: ver 3 1 (tinfo 0xe090cd40)
2022-02-05 21:18:40.201 8380-8470/com.example.modelviewer I/native: I0205 21:18:40.201260    8470 performance_monitor.cc:115] Event: FeatureExtraction is taking too long, it took 116.984ms
2022-02-05 21:18:40.206 8380-8380/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-05 21:18:40.206 8380-8380/com.example.modelviewer E/eglCodecCommon: glUtilsParamSize: unknow param 0x00008919
2022-02-05 21:18:40.252 8380-8495/com.example.modelviewer E/native: E0205 21:18:40.252206    8495 camera_image_stream.cc:270] FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata.
    === Source Location Trace: ===
    third_party/arcore/ar/camera/android/metadata_cache.cc:52
    third_party/arcore/ar/camera/android/metadata_extraction.cc:53

    === Source Location Trace: ===
    third_party/arcore/ar/camera/android/camera_image_stream.cc:204
RGregat commented 2 years ago

Internet permission is added in the Manifest?

<uses-permission android:name="android.permission.INTERNET" />

davidliang104 commented 2 years ago

That did it, thank you! I only copied the permissions from the README and didn't think to check the manifest file too. However, restarting the app while running caused it to crash with this error message, although I'm not sure if this is a different issue.

E/EGL_emulation: eglQueryContext 32c0  EGL_BAD_ATTRIBUTE
    tid 6909: eglQueryContext(1903): error 0x3004 (EGL_BAD_ATTRIBUTE)
E/CameraCaptureSession: Session 0: Exception while stopping repeating: 
    android.hardware.camera2.CameraAccessException: CAMERA_ERROR (3): cancelRequest:456: Camera 0: Error clearing streaming request: Function not implemented (-38)
        at android.hardware.camera2.CameraManager.throwAsPublicException(CameraManager.java:799)
        at android.hardware.camera2.impl.ICameraDeviceUserWrapper.cancelRequest(ICameraDeviceUserWrapper.java:97)
        at android.hardware.camera2.impl.CameraDeviceImpl.stopRepeating(CameraDeviceImpl.java:1138)
        at android.hardware.camera2.impl.CameraCaptureSessionImpl.close(CameraCaptureSessionImpl.java:526)
        at android.hardware.camera2.impl.CameraCaptureSessionImpl$2.onDisconnected(CameraCaptureSessionImpl.java:737)
        at android.hardware.camera2.impl.CameraDeviceImpl$7.run(CameraDeviceImpl.java:239)
        at android.os.Handler.handleCallback(Handler.java:883)
        at android.os.Handler.dispatchMessage(Handler.java:100)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: android.os.ServiceSpecificException: cancelRequest:456: Camera 0: Error clearing streaming request: Function not implemented (-38) (code 10)
        at android.os.Parcel.createException(Parcel.java:2085)
        at android.os.Parcel.readException(Parcel.java:2039)
        at android.os.Parcel.readException(Parcel.java:1987)
        at android.hardware.camera2.ICameraDeviceUser$Stub$Proxy.cancelRequest(ICameraDeviceUser.java:658)
        at android.hardware.camera2.impl.ICameraDeviceUserWrapper.cancelRequest(ICameraDeviceUserWrapper.java:95)
        at android.hardware.camera2.impl.CameraDeviceImpl.stopRepeating(CameraDeviceImpl.java:1138) 
        at android.hardware.camera2.impl.CameraCaptureSessionImpl.close(CameraCaptureSessionImpl.java:526) 
        at android.hardware.camera2.impl.CameraCaptureSessionImpl$2.onDisconnected(CameraCaptureSessionImpl.java:737) 
        at android.hardware.camera2.impl.CameraDeviceImpl$7.run(CameraDeviceImpl.java:239) 
        at android.os.Handler.handleCallback(Handler.java:883) 
        at android.os.Handler.dispatchMessage(Handler.java:100) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
W/CameraDevice(0): Device disconnected.
E/StartedCameraController(0): Device closing due to unexpected device event: DeviceErrorEvent(errorType=ERROR_CAMERA_DISCONNECTED)

Additionally, I wanted to add my own model from the assets folder, would I only need to replace the text in the Uri.parse method? I changed it like so to view a model of around 7MB large,

ModelRenderable.builder()
                .setSource(this, Uri.parse("scene.glb"))
                ...

but only the label appears floating in the air without the model. Changing it to the path from Content Root (src/main/assets/scene.glb) did not create the title or the model. Should something else have been added or removed for local files?

RGregat commented 2 years ago

What are the dimensions of scene.glb? Maybe it is so big that you are inside of the model. You can check the size for example in Blender, or you ask the model to give you the BoundingBox.

// Handle null values 
// getHalfExtent gives you the distance from the center. 
model.getRenderableInstance().getFilamentAsset().getBoundingBox().getHalfExtent();

// First of all you have now a numerical value for the size of the model (halfed values).
// Now you can use the halfed values to calculated a scale if you want.

You can change the scale of the Node where the model is attached to

node.setLocalScale(new Vector3(0.5f, 0.5f, 0.5f));
davidliang104 commented 2 years ago

The dimensions seemed large, but the model was still invisible when adjusting the Node scale or trying to view it in Blender even though it was visible in 3D model viewers, so I just tested different models until one of them worked.

The app still crashes after restarting though, and I'm not sure why. It usually shows the errors I typed earlier followed by these lines before a crash:

I/native: I0210 03:41:50.014537    9250 logger.h:28] DataSourceMetrics: kFirstGlCallback: 1.3006001s
I/native: I0210 03:41:50.055736    9234 timebase_helpers.cc:169] Timebase offset intialized to 0
    I0210 03:41:50.056451    9234 logger.h:28] DataSourceMetrics: kFirstImageCallback: 1.342847s
W/native: W0210 03:41:50.171962    9242 feature_matcher_and_filter_utils.cc:259] INVALID_ARGUMENT: integration window start at 0
    === Source Location Trace: ===
    third_party/redwood/perception/imu_processing/imu_integrator/imu_integrator_utils.cc:96
     Use identity R.
W/native: W0210 03:41:50.185657    9245 motion_tracking_context.cc:1524] Unable to find measurements corresponding to VIO status at timestamp 1486200177100.
I/Choreographer: Skipped 188 frames!  The application may be doing too much work on its main thread.
I/native: I0210 03:41:50.386949    9141 session.cc:3330] Update Frame Delay to 0 frames.
D/EGL_emulation: eglCreateContext: 0xafedd0a0: maj 3 min 1 rcv 4
D/EGL_emulation: eglMakeCurrent: 0xafedd0a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xafedd0a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
W/native: W0210 03:41:50.474278    9141 pose_manager.cc:66] GetRecentDevicePose failed. INVALID_ARGUMENT: Passed timestamp is too old.
    === Source Location Trace: ===
    third_party/redwood/perception/pose_manager/pose_manager.cc:406
     Latest VIO t: -1ns, latest IMU t: 24m46.7124327s, query t:24m45.0542641s
D/EGL_emulation: eglMakeCurrent: 0xafedd0a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xafedd0a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
W/ple.modelviewe: Accessing hidden method Landroid/media/Image$Plane;-><init>()V (greylist, linking, allowed)
I/native: I0210 03:41:51.145361    9241 bundle_adjustment_initializer.cc:268] Intrinsic vector size of the camera 0 is 7
E/native: E0210 03:41:51.180775    9141 session.cc:2517] Invalid ray produced by view data!
I/native: I0210 03:41:51.219823    9229 performance_monitor.cc:115] Event: FeatureExtraction is taking too long, it took 115.013ms
I/native: I0210 03:41:51.279934    9141 session.cc:3330] Update Frame Delay to 0 frames.
D/EGL_emulation: eglCreateContext: 0xafedd3a0: maj 3 min 1 rcv 4
D/EGL_emulation: eglMakeCurrent: 0xafedd3a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
I/native: I0210 03:41:51.302705    9241 bundle_adjustment_initialization.h:136] Number of measurements used in BA initialization for temporal landmarks: 576
    I0210 03:41:51.302878    9241 bundle_adjustment_initialization.h:138] Number of good measurements (i.e., reprojection errors <= 3 pixels) in BA initialization for temporal landmarks: 576
D/EGL_emulation: eglMakeCurrent: 0xafedd3a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xafedd3a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xafedd3a0: ver 3 1 (tinfo 0xd78a6a30)
D/EGL_emulation: eglMakeCurrent: 0xd785b600: ver 3 1 (tinfo 0xd78a6a30)
E/native: E0210 03:41:51.433258    9141 session.cc:2517] Invalid ray produced by view data!

I thought it might be because some part of the session was still in the memory, so I added methods like arFragment.getArSceneView().destroySession() and arFragment.getArSceneView().destroy() to onDestroy(), but that didn't work.

github-actions[bot] commented 1 year ago

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 year ago

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please feel free to create a new issue with up-to-date information.