search

Samsung / GearVRf

The GearVR framework(GearVRf) is an Open Source VR rendering library for application development on VR-supported Android devices.
http://www.gearvrf.org
Apache License 2.0
407 stars 217 forks source link

GVRViewSceneObject not working after migration from 3.2 to 4.0 #1944

Closed felislynx-silae closed 6 years ago

felislynx-silae commented 6 years ago

Hi :) Finally i'm migrating from old framework to new one. After migrating, old code that was showing Android Views is not working. Code in 3.2

gvrContext.activity.runOnUiThread {
            val frameLayout = GVRFrameLayout(gvrContext.activity)
            View.inflate(gvrContext.activity, R.layout.left_panel, frameLayout)
            frameLayout.setBackgroundColor(Color.BLACK)
            frameLayout.layoutParams.width = 750
            frameLayout.layoutParams.height = 300
            val sceneObject = GVRViewSceneObject(gvrContext,frameLayout,gvrContext.createQuad(1.5f,1f))
            sceneObject.transform.positionZ = -2.5f
            sceneObject.transform.positionX = -1f
            sceneObject.transform.positionY = -1f
            addSceneObject(sceneObject)
        }

Code in 4.0

gvrContext.activity.runOnUiThread {
            val frameLayout = FrameLayout(gvrContext.activity)
            gvrContext.activity.registerView(frameLayout)
            View.inflate(gvrContext.activity, R.layout.left_panel, frameLayout)
            frameLayout.setBackgroundColor(Color.BLACK)
            frameLayout.layoutParams.width = 750
            frameLayout.layoutParams.height = 300
            gvrContext.activity.unregisterView(frameLayout)
            val sceneObject = GVRViewSceneObject(gvrContext,frameLayout,gvrContext.createQuad(1.5f,1f))
            sceneObject.transform.positionZ = -2.5f
            sceneObject.transform.positionX = -1f
            sceneObject.transform.positionY = -1f
            addSceneObject(sceneObject)
        }

In 3.2 i could see android view and it was working. In 4.0 i don't see this view. I've checked in debugger that view is created, inflated, but scene looks empty. Did i missed something during migration for GVRViewSceneObject?

liaxim commented 6 years ago

Hi @felislynx-silae . Converted gvr-events to use your code and do this from onInit:

        gvrContext.getActivity().runOnUiThread(
                new Runnable() {
                    public void run() {
                        frameLayout = new FrameLayout(gvrContext.getActivity());
                        gvrContext.getApplication().registerView(frameLayout);
                        View.inflate(gvrContext.getActivity(), R.layout.activity_main, frameLayout);
                        frameLayout.setBackgroundColor(Color.WHITE);
                        frameLayout.getLayoutParams().width = 750;
                        frameLayout.getLayoutParams().height = 300;
                        gvrContext.getApplication().unregisterView(frameLayout);

                        GVRViewSceneObject sceneObject = new GVRViewSceneObject(gvrContext, frameLayout, gvrContext.createQuad(1.5f, 1f));
                        sceneObject.getTransform().setPositionZ(-2.5f);
                        sceneObject.getTransform().setPositionX(-1f);
                        sceneObject.getTransform().setPositionY(-1f);

                        gvrContext.getMainScene().addSceneObject(sceneObject);
                    }
                }
        );

It worked as expected for me. Do you notice any suspicious logcat messages?

felislynx-silae commented 6 years ago

@liaxim what tag i should look at?

felislynx-silae commented 6 years ago

I've runned app in developer mode and those are only "not ordinary outputs"

07-16 17:51:55.512 7733-7785/pl.fream.evryplacevr.gearvr.evr.debug W/gvrf: GVRActivity::leaveVrMode: ignored, have not entered vrMode
07-16 17:51:55.970 7733-7808/pl.fream.evryplacevr.gearvr.evr.debug W/VRSurfaceManager: VRSurfaceManager : setFrontBuffer
07-16 17:51:58.913 7733-7785/pl.fream.evryplacevr.gearvr.evr.debug E/GVRShaderTemplate: SHADER: generated shader #1 GVRTextureShader$a_texcoord
07-16 17:51:58.918 7733-7785/pl.fream.evryplacevr.gearvr.evr.debug E/GVRShaderTemplate: SHADER: generated shader #2 GVRTextureShader$u_texture-#a_texcoord#
07-16 17:51:58.923 7733-7785/pl.fream.evryplacevr.gearvr.evr.debug E/GVRShaderTemplate: SHADER: generated shader #3 GVROESShader
felislynx-silae commented 6 years ago

By the way: Gradle dependencies implementation 'org.gearvrf:framework:4.0.0' implementation 'org.gearvrf:3DCursorLibrary:4.0.0' implementation 'org.gearvrf:backend_oculus:4.0.0'

liaxim commented 6 years ago

Can you try 4.0.1-SNAPSHOT instead of 4.0.0?

felislynx-silae commented 6 years ago

Sure, it'll take a while. 4.0.1 have more breaking changes to my code. I'll update you ASAP

liaxim commented 6 years ago

If you are going through all the trouble to migrate, it is better to migrate to 4.0.1.

felislynx-silae commented 6 years ago

@liaxim I've made all changes but last thing that prevents me from run and check if views are visible:

Execution failed for task ':app:transformNativeLibsWithMergeJniLibsForEvryplaceOculusDebug'.
> More than one file was found with OS independent path 'lib/armeabi-v7a/libc++_shared.so'
liaxim commented 6 years ago

Add the following packaging options:

    packagingOptions {
        pickFirst 'lib/*/libc++_shared.so'
    }
felislynx-silae commented 6 years ago

It works :) I can see my view. Do you recommend some demo to look at new way to implement cursor events? CursorActivationListener, CursorActivationListener are missing now so i believe they are replaced by new API

liaxim commented 6 years ago

You could look at gvr-controller.

felislynx-silae commented 6 years ago

Thank you very much. I hope it'll work with Oculus GO too ;)

felislynx-silae commented 6 years ago

@liaxim Is it allowed to change mainScene or in 4.0.1 this will not work

liaxim commented 6 years ago

@felislynx-silae Still works

felislynx-silae commented 6 years ago

I've get this exception after changing scene and using assetloader to load android resource (background photosphere)

07-17 13:06:16.292 31134-31297/? A/libc: Fatal signal 11 (SIGSEGV), code 2, fault addr 0x76c1f80a in tid 31297 (GLThread 5775)
07-17 13:06:16.519 32740-32740/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    Build fingerprint: 'samsung/zerofltexx/zeroflte:7.0/NRD90M/G920FXXU6ERD1:user/release-keys'
    Revision: '11'
    ABI: 'arm'
    pid: 31134, tid: 31297, name: GLThread 5775  >>> pl.fream.evryplacevr.gearvr.evr.debug <<<
    signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x76c1f80a
        r0 b7444000  r1 76c1f80a  r2 00000010  r3 00000000
        r4 ef7b5008  r5 cabd86e8  r6 cabd86f8  r7 cabd8b30
07-17 13:06:16.520 32740-32740/? A/DEBUG:     r8 cabd871c  r9 cabd8704  sl cabd8b20  fp 00000000
        ip cd422c38  sp cabd8588  lr cbc2b460  pc ef73f610  cpsr 000f0010
07-17 13:06:16.538 32740-32740/? A/DEBUG: backtrace:
07-17 13:06:16.539 32740-32740/? A/DEBUG:     #00 pc 00017610  /system/lib/libc.so (memcpy+156)
        #01 pc 0032845c  /system/vendor/lib/egl/libGLES_mali.so
        #02 pc 003287e8  /system/vendor/lib/egl/libGLES_mali.so
        #03 pc 0066a210  /system/vendor/lib/egl/libGLES_mali.so
        #04 pc 0068957c  /system/vendor/lib/egl/libGLES_mali.so
        #05 pc 0068abd4  /system/vendor/lib/egl/libGLES_mali.so
        #06 pc 00677634  /system/vendor/lib/egl/libGLES_mali.so
        #07 pc 00678200  /system/vendor/lib/egl/libGLES_mali.so
        #08 pc 00608090  /system/vendor/lib/egl/libGLES_mali.so (glCompressedTexImage2D+156)
        #09 pc 001271c9  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr13GLBitmapImage16updateFromMemoryEi+180)
        #10 pc 00126f57  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr13GLBitmapImage6updateEi+66)
        #11 pc 001218c5  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr5Image14checkForUpdateEi+36)
07-17 13:06:16.540 32740-32740/? A/DEBUG:     #12 pc 00147703  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr7Texture7isReadyEv+18)
        #13 pc 00137bb7  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr10ShaderData9updateGPUEPNS_8RendererEPNS_10RenderDataE+46)
        #14 pc 00134c59  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr10RenderPass7isValidEPNS_8RendererERKNS_11RenderStateEPNS_10RenderDataE+70)
        #15 pc 00141f1f  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr10RenderData7isValidEPNS_8RendererERKNS_11RenderStateE+110)
        #16 pc 001226f7  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr8Renderer13addRenderDataEPNS_10RenderDataERNS_11RenderStateERNSt6__ndk16vectorIS2_NS5_9allocatorIS2_EEEE+30)
        #17 pc 001227dd  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr8Renderer19occlusion_cull_initERNS_11RenderStateERNSt6__ndk16vectorIPNS_11SceneObjectENS3_9allocatorIS6_EEEEPNS4_IPNS_10RenderDataENS7_ISC_EEEE+164)
        #18 pc 00120a9f  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr10GLRenderer14occlusion_cullERNS_11RenderStateERNSt6__ndk16vectorIPNS_11SceneObjectENS3_9allocatorIS6_EEEEPNS4_IPNS_10RenderDataENS7_ISC_EEEE+38)
        #19 pc 0012265d  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr8Renderer14cullFromCameraEPNS_5SceneEP8_jobjectPNS_6CameraEPNS_13ShaderManagerEPNSt6__ndk16vectorIPNS_10RenderDataENS9_9allocatorISC_EEEEb+792)
        #20 pc 00142a45  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (_ZN3gvr12RenderTarget14cullFromCameraEPNS_5SceneEP8_jobjectPNS_6CameraEPNS_8RendererEPNS_13ShaderManagerE+40)
        #21 pc 00143149  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/lib/arm/libgvrf.so (Java_org_gearvrf_NativeRenderTarget_cullFromCamera+88)
        #22 pc 01331ce1  /data/app/pl.fream.evryplacevr.gearvr.evr.debug-2/oat/arm/base.odex (offset 0x1218000)

Also i hve more detailes crashlog (700 lines of native exception)

felislynx-silae commented 6 years ago

GOLog.txt

felislynx-silae commented 6 years ago

Interesting thing is that if i dont call this to setup background sphere in first scene, call in second sphere results in loading background. If i go back to first scene and call this, it results in black texture. (Or if i call it on first scene and then on second, second one have black texture). Third call on third scene causes crash of GL thread

        val texture =
            gvrContext.assetLoader.loadTexture(GVRAndroidResource(gvrContext, "phl.astc"))
        sphere.renderData.material.mainTexture = textureLeft
felislynx-silae commented 6 years ago

@liaxim i've made change, that on one single scene if i click a button, new texture is loaded for sphere (i'm rotating 3 different images). Results -> Initial launch (without texture) white background 1click -> proper texture 2click -> after a while second texture is loading 3click -> background is black 4click -> background is black 5click -> sometimes texture is loaded 6click -> black background

In most cases, after first click, asset loader stops loading properly any textures. If i do this after changing a scene asset loader causes crashes described above

felislynx-silae commented 6 years ago

Another insight: after modifying code to load 4 sphere textures inside Main onInit (before setting main scene), i can swap texture and it works correctly. This breaks functionality of downloading assets from web and setting them as background. I think that on 4.0.1 there might be something with assetloader

liaxim commented 6 years ago

@felislynx-silae Will look into it. It will be quicker if you could share some sample app that reproduces the problems.

felislynx-silae commented 6 years ago

Sure, i'm extracting necessary code right now

felislynx-silae commented 6 years ago

@liaxim Murphy's law. Very very simple code, works o_O. https://github.com/felislynx-silae/GearBugSample (develop) This is simplified version which doesn't inflate all layouts, do network calls. But workflow, threading is the same. (code is copy pasted so it is not different from main app) I'll add more code until i'll recreate issue. Damn murphy's laws.

liaxim commented 6 years ago

There are two different crashes in the logs. It is clear that invalid memory access is occurring. Switching the scenes leads to dangling pointers left around somewhere.

liaxim commented 6 years ago

What is the memory usage of your app? When you switch the scene do you keep references to the previous scene? In other words, do you think a lot of garbage collection goes on?

felislynx-silae commented 6 years ago

I don't think so. On 3.2 we could use app for 40 minutes without any issues :) also with spherical videos. Before spike of network there is Menu -> Loader and after spike Loader -> Presentation memory usage1 memory usage2

felislynx-silae commented 6 years ago

@liaxim second crash from txt file is present only on 4.0.1 (JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal start byte 0xaa). This is happening while saving Retrofit body to a file (astc file is compressed texture). On 3.2 no problems, on 4.0.1 some spheres are causing this issue :). I've read that this might be due to 32/64 libs. But no good solution to avoid this. Right now i've patchworked around this. 

val fileReader: ByteArray? = ByteArray(4096)
inputStream = body.byteStream()
outputStream = FileOutputStream(targetFile, append)
while (true) {
val read = inputStream?.read(fileReader) ?: -1 //<-- this line is causing crash
if (read == -1) {
break
}
outputStream.write(fileReader, 0, read)
}
outputStream.flush()
liaxim commented 6 years ago

Tried to reproduce your problem by frequently changing the main scene and adding photospheres using background threads. No luck.

The UTF-8 error I believe is a symptom of memory corruption actually. It happens while accessing the filename associated with a GVRImage. It looks to me a bad pointer to a BitmapImage is being used. The format doesn't matter when it comes to the filename.

Do some of the images you load from the network take long time? Like tens of seconds.

felislynx-silae commented 6 years ago

Some of them may take a long time, but GVRImage is not used until file is saved. method above must have provide callback to scene, that it is safe to use an image. It wasn't a problem before :). Would it help if i would provide obfuscated APK where the problem occurs?

liaxim commented 6 years ago

It might. Send it to m.marinov@samsung.com.

felislynx-silae commented 6 years ago

@liaxim i've forgot about one thing... OSIG

felislynx-silae commented 6 years ago

When i get "Texture X loaded from cache" black screen occurs:

07-20 08:39:45.754 18713-18713/pl.fream.evryplacevr.gearvr.evr.debug V/ASSET: loadTexture phl.astc
    loadTexture phr.astc
07-20 08:39:45.755 18713-19008/pl.fream.evryplacevr.gearvr.evr.debug V/ASSET: Texture: phl.astc loaded from cache
07-20 08:39:45.757 18713-19003/pl.fream.evryplacevr.gearvr.evr.debug V/ASSET: Texture: phr.astc loaded from cache
felislynx-silae commented 6 years ago

I've updated github project, bug is reproducible on S6. it depends how many different textures i'll load.

liaxim commented 6 years ago

The particular crash in the latest GearBugSample is a problem in our Java/C++ objects lifecycle management. It looks to be raised in GVRCursorController.setScene which reacts to main scene changes.

You could easily work around it by not switching the scenes. I am looking at it but have no meaningful progress.

liaxim commented 6 years ago

Have a fix for you. Wouldn't be surprised it is not the only one you need.

felislynx-silae commented 6 years ago

@liaxim do you think this will also fix the black texture issue from my commit: 4a0d519e9493b1ba630081cc957492b1580796d0? I've reproduced it 2 times ;) one time with switching scenes and commit before (the one with hash above) without, just switching textures (but loading many different textures, not just swapping 2).

liaxim commented 6 years ago

@felislynx-silae No. Will check it out.

liaxim commented 6 years ago

Checked the latest GearBugSample.

There is a reference to 3.JPG when the file is named p3.JPG. Also phl.astc and phr.astc do not seem to be valid ASTC-compressed files.

felislynx-silae commented 6 years ago

I've used https://github.com/ARM-software/astc-encoder with this command astcenc -c ./phl.jpg ./phr.astc 6x6 -medium I'm correcting 3.JPG and i'll try again to reproduce

felislynx-silae commented 6 years ago

@liaxim i can confirm, that using JPG's works. It is strange that ASTC is not working now on new lib. On previous version using those ASTC's gave us enormous performance boost.

liaxim commented 6 years ago

Decoding to jpeg with the Mali tools failed for me with possible corruption error. Also the astc file is surprisingly large. If you include the jpg's too in the repo I will try encoding them also. We will get astc working if there is a problem.

felislynx-silae commented 6 years ago

@liaxim Pushed to github. Until this is somehow resolved i've switched my development version to JPG's to finish up upgrading. Production version is using ASTC (with 3.2 framework)

liaxim commented 6 years ago

It doesn't look to be an inherent problem in our compressed images support. More like an issue with the Image/Texture classes in the scenario your app exercises.

liaxim commented 6 years ago

Traced it to a problem with the texture parameters. We have a mIsCompressed flag that looks to be reset (or never set) correctly whenever the texture is created again and again for the same image. We cache the so called images but not the so called texture objects.

liaxim commented 6 years ago

@felislynx-silae https://github.com/Samsung/GearVRf/pull/1963 fixes your problem.

liaxim commented 6 years ago

@felislynx-silae Fix has been merged. Snapshots updated too. Please try again.

felislynx-silae commented 6 years ago

@liaxim i've checked latest snapshot:

W/System.err:     at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)
        at java.io.FileInputStream.<init>(FileInputStream.java:99)
        at org.gearvrf.utility.MarkingFileInputStream.<init>(MarkingFileInputStream.java:34)
        at org.gearvrf.GVRAndroidResource.openStream(GVRAndroidResource.java:377)
        at org.gearvrf.GVRAndroidResource.getCompressedLoader(GVRAndroidResource.java:671)
        at org.gearvrf.asynchronous.GVRAsynchronousResourceLoader$2.run(GVRAsynchronousResourceLoader.java:205)
        at org.gearvrf.utility.Threads$1.run(Threads.java:108)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:428)
liaxim commented 6 years ago

@felislynx-silae Could you please provide further details? Are you getting a FileNotFoundException?