Galaxy S7 Edge US edition performance issues - Submission

[felislynx-silae]

Hi, I've quite large question. Since feature development of my product is on hold until first release on Oculus store will be achieved, so i'll get back to old questions/issues after submission will be complete. But i'm struggling with it since september last year (feedback from oculus team is poor, so it is long process). They are failing my submission because on Galaxy S7 Edge (snapdragon version) there is lots of frame drops, judder etc. I've S7 Edge with Exynos processor, and it is working like a charm. Do you encountered that issue? Buying another phone from US series would be troublesome. I've saw, that Unity/Unreal devs had similar problems. I'm on 3.2 version of GearVRf (since further releases had many breaking changes and buisness wanted to release on this version and then upgrade to new one). All my textures are ASTC's for better performance (on our test phones, result is phenomenal!). But still, S7Edge snapdragon is failing to work as expected.

[bohuang3d]

That could be from many factors and processing demand of your app. What is the frame rate on either?

Please share your per frame triangle count draw call count time spent in physics/ai/rendering/post effects/etc. memory usage number of textures advanced techniques etc.

Then we can figure something out

[felislynx-silae]

@bohuang3d I can't provide you that informations. Oculus testing team didn't provide those. As i wrote earlier on all my devices (S6,S7,S7Edge,S8) app performs perfectly. But i have european version of those phones. Oculus testing team obviously is using US version. App is not very demanding for processing power. It is designed to provide VR tours via photospheres. So this means, no physics, ai, post effects.

[bohuang3d]

Can you run the sample photosphere app from GearVRF and report if there is performance difference?

[felislynx-silae]

I dont own us version of s7 edge. Oculus testing team does. https://forums.oculusvr.com/community/discussion/59954/help-needed-failing-submission-because-of-judder-on-s7-edge-with-snapdragon-all-others-are-ok https://forums.oculusvr.com/developer/discussion/54714/your-app-exhibits-judder Unity and unreal have problems with this particular phone as you can read on those websites. All of my test users are from Europe so we dont experience any issues with app. But oculus submission process takes tests on Galaxy S7 Edge 6.0 us version. And submission fails because of that. That's why i asked if any of you, might experienced issues on this phone and or have solution.

[bohuang3d]

I wanted to check if this is a GearVRf problem. Sounds like it's not. Rather, it's a device specific problem.

Usually, it's best as a developer to have an idea of the aforementioned statistics to see what's spending your cpu/gpu times. So you can have a more targeted approach.

Nonetheless, can you send me an email with your app's description and screenshots

[thomasflynn]

Hi there. If you're in the submission pipeline, then you probably have a beta channel in which you can add users to be able to download and test your application, right? If so, please add both: flynnt and flynnie

Also @bohuang3d and Nite, please give @felislynx-silae your IDs.

If we can download the app, we can probably run it in the SnapDragon Profiler and try to get a better clue as to where the bottleneck is.

Finally, perhaps bumping up the CPU or GPU level will do the trick? Like set the GPU level to 3? we default to 2/2, but you can override that in your gvr.xml file.

[felislynx-silae]

@thomasflynn if you send me your email on oculus and also @bohuang3d, i'll send invitation to beta channel on oculus store :). By the way, currently my params in gvr.xml

<lens name="N4" >

    <vr-app-settings
        framebufferPixelsHigh="DEFAULT"
        framebufferPixelsWide="DEFAULT"
        showLoadingIcon="true"
        useProtectedFramebuffer="false"
        useGazeCursorController="true"
        useSrgbFramebuffer="false" >

        <mono-mode-parms
            monoFullScreen="false"
            monoMode="false" />

        <mode-parms
            allowPowerSave="true"
            resetWindowFullScreen="true" />

        <performance-parms
            cpuLevel="1"
            gpuLevel="1" />

        <eye-buffer-parms
            colorFormat="COLOR_8888"
            depthFormat="DEPTH_0"
            fov-y="90.0"
            multiSamples="4"
            resolutionWidth="1280"
            resolutionHeight="1440"
            resolveDepth="false" />

        <head-model-parms
            eyeHeight="DEFAULT"
            headModelDepth="DEFAULT"
            headModelHeight="DEFAULT"
            interpupillaryDistance="DEFAULT" />
    </vr-app-settings>

</lens>

[thomasflynn]

tom.flynn@samsung.com and flynnt@acm.org. And, yeah, you have cpu/gpu set to 1/1. you should be able to set that to 2/2 without trouble.

[felislynx-silae]

@thomasflynn done :)

[thomasflynn]

Ok, thanks for the invite. I've been able to download and play with it on a Snapdragon S7 Edge running Android 7 (Nougat).

First, good job :). The stereo images look very nice and the user interface looks and works very well. Good work :)

You mentioned that Oculus was complaining about "Galaxy S7 Edge 6.0 us version". Does that mean Android 6.0? as in Marshmallow? I'm assuming so. We grabbed an S7 Edge and flashed it with 6.0, but I am unable to see/install your app. It does not show up in the 'My Preview Apps' section like it does with the S7 phones running Nougat (Android 7). If I take the apk and try to install it manually with adb, I get: Failure [INSTALL_FAILED_OLDER_SDK]

What's your minSdkVersion set to? Maybe Oculus is faling it because it won't install on Marshmallow?

[apjagdale]

Hi @felislynx-silae I notice one thing, you are using lot of custom shader(CS). If I enter a room CS is added, move to another room CS is added. Again moving to the previous room CS is added. If you have few CS, you can just create them once and reuse them.

[felislynx-silae]

@thomasflynn funny thing about 6.0. We thought, that because we have 7.0 and it works perfectly, and even SGS6 have 7.0, then we will cut out everything below api 24 (latest submission attempt). And after that, submission status was still failed because of SGS7E, i've started digging around and voila US/EU version difference. Before cutting out older api, we had exactly same "issues" with submission. I'll give you all feedback that i've got: VRC.Mobile.Input.3 ->

Notes from Reviewer

Short press back button works fine until the Main Menu where it should bring up the "Exit to Oculus Home screen?" where the user can have the option to cancel or Go to Oculus Home.
Steps to Reproduce

1. Launch the title
2. If applicable, select a menu item to begin the experience
3. Press the back button
4. Observe the display
5. Press the back button again
Expected Results

If the app has a pre-launch menu, pressing the back button once should take you back to it.  Pressing the back button multiple times should eventually show the "Return to Oculus Home" menu.

VRC.Mobile.Performance.1 ->

Notes from Reviewer

Still jitter and judder issues on the S7 Edge. Occurs on all our S7 devices.
Steps to Reproduce

1. Use the app for the length of the content or 45 minutes, whichever is shorter
2. Launch the OVR Metrics tool and select the log file for your app
3. Observe the FPS graph
Expected Results

Application should not experience extended periods of framerate below 60 FPS. Exceptions include when there's a black screen or during loading scenes.

VRC.Mobile.SDK.1 ->

Notes from Reviewer

Still jitter and judder issues on the S7 Edge. Occurs on all our S7 devices.

After that submission test result i just had to ask here if anyone had issues with SGS7Edge before since i saw lots of topics about poor performance of US version.

@apjagdale For every widget responsible for transition into another scene, we add shader which is responsible for animation around icon. We are changing values inside this shader to animate widget. I could try to have one instance but i would have to test, how it would behave if we would have ie. 10 transition widgets on scene :)

[liaxim]

@felislynx-silae Two things:

1. You could try lowering the eye buffer's resolution to the default 1024 (in gvr.xml)

           resolutionWidth="1024"
           resolutionHeight="1024"

2. You could enable multiview for your app; you will have to adjust your shaders, which we can help with. In gvr.xml:

<lens name="N4" >
...
    <vr-app-settings
        ...
        useMultiview="true"

@apjagdale Could you please try to run the app for extended period of time? I guess up to 45 minutes is to be expected. Please also figure out the cpu usage pattern.

[felislynx-silae]

@liaxim about eye buffer resolution, we increased it just because users were telling us, that for 1024x1024 value quality was worse than webview+cardboard :) We are still trying to get hold on us version of this phone. But i'll prepare version with reuse of shaders and useMultiview to check out performance gain

[apjagdale]

@felislynx-silae Regarding reusing Custom Shaders, we cache it in framework. I am unaware as why your app is creating Custom Shaders all the time. We cache CS by its class name. You can check our gvr-eyepicking demo which uses CS.

[felislynx-silae]

@apjagdale could it be connected to use of ASTC textures for photospheres?

[felislynx-silae]

New version is uploaded into Beta channel.

• shaders for transition objects are now reused instead create over and over and over again
• i've added .nomedia to root folder of app to prevent media scan for new thumbnails and assets
• i've raised cpu level to 2

[bohuang3d]

@felislynx-silae I was seeing some network activities while at the start menu interface doing nothing. There seems to be consistent network activities and sometimes relatively high.

Do you know what's going on there?

[thomasflynn]

That sounds like something to look into. That aside however, there has been a lot of improvement. We can verify that shaders are no longer being recompiled. And framerate is a lot more steady. Performance on Snapdragon and Exynos looks pretty similar (with Snapdragon looking slightly better).

[felislynx-silae]

We have submitted new version to review. Fingers crossed :) The network activity is actually download jobs that are working while app is idle (ones that didn't have a chance to be finished while watching presentation). It is a requested feature.

[felislynx-silae]

So, again they said that 1. S7 edge has judder and 2. Short back navigation is wrong. And honestly i dont see what is wrong with back navigation now. And why there is still some kind of judder on their s7 edge. They didnt provide any additional information...

[thomasflynn]

can they share the log of the run with you?

[felislynx-silae]

I've made ticket with request for logs. I'll send them as soon as ill get them

[bohuang3d]

@felislynx-silae Do you mean you are downloading bits and pieces of the 360 degree photographs while idle?

[felislynx-silae]

@bohuang3d more or less. Some parts that will speed up future run. It make sense for our target user. Maybe not for typical user that will download this app once, run for 30 minutes and then unistall :)

[liaxim]

@felislynx-silae Can't you download all that stuff prior to the first launch? Like show loading asset notice and launch only after done. All that activity could be contributing to the jitter the tester is seeing. Do you throttle all those downloads or do they happen as fast as the network and the device allow? How big of a download are we talking about? How big are the files, how many? What happens if the phone is not connected to a network? Do you do something with those files immediately after downloading?

[felislynx-silae]

Those are 4k spheres and we dont download them in pre launch since some presentations weight a lot and it is better to fast launch it with 1k sphere and then download on demand 4k per visited scene (typical sphere 4k is ~4mb). Throttle is set to 5 files at once. Quantity depend how many scenes you visited and exited before 4k loads. They are saved just like any other asset. If phone is disconnected from the network work is disposed and if user will go for this asset again, download will resume. If we are in main menu, after file is downloaded, nothing happens. It is just ready for another run of this presentation :). I can disable this feature and check on ovr metrics tool for performance changes tommorow morning :)

[bohuang3d]

@felislynx-silae

You say that others have reported problems with the Snapdragon S7 as well. Due to chip architecture differences, it’s a possibility that the Qualcomm chipset may not handle some operations as well as the Euro ARM chipset, thus exposing the vulnerable parts of your app using those operations. (On the other hand the Snapdragon chipset may excel in other areas compared to ARM)

That speculation aside, the S7 is a sailed ship and your app has to adapt to its quirks. I would want you to log some performance metrics and do some sanity checks. For example, is it reasonable for the app to download 3kb to 18kb of data each frame? How long does it take to parse the data? The length of time to transform the data into other images/data structures? Any bottleneck during this chain of processes could be the cause of the occasional frame rate drops.

[felislynx-silae]

So after next submission, they didn't failed it because of SGS7Edge so it means that background network shouldn't be done while idle :). Also shader was an issue. Although they still fail us because of back navigation. I believe that this is because when you go back from presentation into main menu, first back event is doing nothing. Which is quite strange

override fun dispatchKeyEvent(event: KeyEvent): Boolean {
        return if (applicationMain.onBackPressed(event)) { // If this is not MenuScene, event will be consumed and returns true, MenuScene returns false and event is not consumed
            true
        } else {
            super.dispatchKeyEvent(event)
        }
    }

I'm looking why first super.dispatchKeyEvent(event) gives no effect and second one is invoiking "Exit to oculus screen". Is there a way to invoke this menu by code in framework?

[bohuang3d]

Great to hear. The rest of your issues should hopefully be easier comparatively.

For the back navigation, I can't tell the logic flow of this 'return if' block. I'm guessing this function returns true if applicationMain knows if back key pressed. Otherwise return whatever super.dispatchKeyEvent(event) returns.

Can you debug and examine the 'event' object. What action(s) correspond to the first super.dispatchKeyEvent(event) you mention?

What is the shader issue

[felislynx-silae]

I've send another set of invitations (we had to change package name & keystore to sign app so new entry in store was created). Today news: only thing they say that is wrong:

Notes from Reviewer

The back button short-press still is not functioning as intended.
Your navigation currently is:
Main Menu -> Presentation ->(back press) Main Menu ->(back press) Nothing happens, you remain on the Main Menu. This is when the "Exit to Oculus Home Screen" should appear but currently it doesn't.
Steps to Reproduce

1. Launch the title
2. If applicable, select a menu item to begin the experience
3. Press the back button
4. Observe the display
5. Press the back button again
Expected Results

If the app has a pre-launch menu, pressing the back button once should take you back to it.  Pressing the back button multiple times should eventually show the "Return to Oculus Home" menu.

Although, on S6, S7 and S7Edge (white and black gogles) we couldn't reproduce this behaviour which is quite strange and thats why i've asked that question. applicationMain knows if scene consumed event (so we shouldn't dispatch it to framework). On Main menu dispatchKeyEvent is dispatching back button keycode with DOWN and UP action correctly, but first one is not doing anything, second one is entering universal menu. I've made some workaround within version 4 but aparently its still wrong

[liaxim]

We have GVRMain.onBackPress to deal with the back key navigation. Currently only gvr-immersivepedia uses it as far as I can recall. If your app decides it wants to handle the back key (to move to a different scene for example) then you must return true from onBackPress. Otherwise the framework handles it. For Oculus we bring up the exit-to-Oculus-home activity. There is no public API to force the Oculus prompt.

Not sure whether you are relying on onBackPress alone. Do you have a custom dispatchKeyEvent?

[bohuang3d]

@felislynx-silae I cannot reproduce on S7edge the way I understand it. At main menu where the scenes are displayed, pressing back I do see the 'return to Oculus Home' screen. Pressing back again brings me back to your main menu window.

You say you can't reproduce either. Which device did the reviewer flag the problem?

[felislynx-silae]

@bohuang3d They are right: Black goggles + s6 -> back button is not working o_O action:0/1 keycode:4 On white gogles same action and keycode but back works

If i leave back button entirely to framework, it is not invoking exit to oculus screen with S6 and black gogles. Scratch that, on white goggles, back works correctly, on black one don't. I've checked even older builds where i didn't intercepted key press. Framework is just not launching "ExitToOculusHome/Cancel" for black goggles

2018-02-15 11:31:47:552 - KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2072339, downTime=2072339, deviceId=21, displayId=0, source=0x2002 }
2018-02-15 11:31:47:726 - KeyEvent { action=ACTION_UP, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2072483, downTime=2072483, deviceId=21, displayId=0, source=0x2002 }
2018-02-15 11:31:49:178 - KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2073995, downTime=2073995, deviceId=21, displayId=0, source=0x2002 }
2018-02-15 11:31:49:316 - KeyEvent { action=ACTION_UP, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2074123, downTime=2074123, deviceId=21, displayId=0, source=0x2002 }
2018-02-15 11:31:49:662 - KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2074483, downTime=2074483, deviceId=21, displayId=0, source=0x2002 }
2018-02-15 11:31:49:796 - KeyEvent { action=ACTION_UP, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=2074594, downTime=2074594, deviceId=21, displayId=0, source=0x2002 }

This is what back button produces on black goggles which is passed to GearVRf

Also i've tried to move logic from dispatchKeyEvent to Main.onBackPress. And result is the same, white goggles are good, developer mode is good, black goggles oculus menu not invoked. I've tried to find out intent that framework is calling to launch this screen but i can't find it :/

I've tried to hack this intent to call it all by myself:

val intent = Intent()
        intent.`package` = "com.oculus.systemactivities"
        intent.component = ComponentName("com.oculus.systemactivities","com.oculus.systemactivities.PlatformActivity")
        intent.putExtra("intent_cmd","confirmQuit")
        intent.putExtra("intent_pkg",baseActivity.applicationContext.packageName)
        intent.addFlags(Intent.FLAG_ACTIVITY_NO_ANIMATION)
        intent.data = Uri.parse("")
        baseActivity.startActivity(intent)
        baseActivity.overridePendingTransition(0, 0)

It is launching oculus home without asking if i want to exit or cancel :/

[liaxim]

@felislynx-silae How does gvr-immersivepedia work for you? Is it too not bringing up the prompt?

[felislynx-silae]

@liaxim Yup it works, but i have version of immersivepedia with 4.0 framework. My app is on 3.2 and update to newer framework before release is impossible

[liaxim]

Ok. Does the 3.2 version work for you?

[felislynx-silae]

On my app, apparently not :). Funny thing is, everything works superb on white goggles or in dev mode. On black goggles from S7 back events are fired, but universal menu is never called. I've checked that on app with removed all "sniffers" to back press or keyevents to make sure, that i'm not messing with events in some way.

Changes done in releases >=3.3 are to damaging to existing code base to do it as "fast change". Migration to 4.0 was put on hold, on november 2017 to finish 1.0 release (it took me almost a month to regain basic functionality on 4.0). Thats why migration to never version is very not welcomed right now. I'll finish changes to immersivepedia to migrate it to 3.2 framework and i'll tell you tomorrow morning.

[felislynx-silae]

@liaxim https://github.com/felislynx-silae/GearBugSample/tree/develop i've used this project to test this bug. Apparently on 3.2 black goggles are not working correctly. If you change to 3.3.0 they are working. So is there any chance to fix this without updating to 3.3 or we have to do it?

[felislynx-silae]

@liaxim Ok, i've made some changes to run app on 3.3.0 (back is working on this version) While my testers are checking debug build to see if nothing has been broken, on release build PlatformEntitlementCheck is failing always:

PlatformEntitlementCheck.start(
                    gvrContext,
                    baseActivity.resources.getString(R.string.APP_ID),
                    entitlementCheckListener)

throws: class java.lang.IllegalStateException Could not initialize the platform sdk; error code: -1

Without Entitlement check Oculus wont publish app :/ EDIT: 3.3 introduced to many bugs to change:

• cursor is not always accessible from cursorManager so most of views is not reacting in that case
• widgets with text are not loading properly
• images are not loading properly
• scene animations are broken

<-- That's why i would prefer to fix it on 3.2 and then jump into 4.0

[liaxim]

Ok, updated 3.2 at https://github.com/liaxim/GearVRf/tree/32update

6d5c2fecac3a0959ece0020a6a1594e7d9e026c4 is the commit you really need. 9e8995051b28af5efb7e79f5236838d504409ec5 you may need to build with newer ndks. 4e5d24cd5d6f461c7bcf47adfea0478f46813ea1 is a nice to have since it makes sure threads get correct priorities.

[bohuang3d]

@felislynx-silae

Before you do another submission, -Make sure the language is English legible. Oculus may fail you for difficult navigation with Polish texts -If possible show with more content. Five to ten environments could be relatively sparse in Oculus' considerations.

If you send me an updated build, my side can give you a look-over to hopefully prevent additional roadblocks from Oculus submission

Edit: And by the way, are you aware that none of the scenes load? Selecting any one the app quits. Log attached evryplace_error_log.txt

[felislynx-silae]

@bohuang3d i'll send as soon as i'll build a new version of 3.2 from liaxim. Yeah i know it is crashing, i've changed package name of rest objects and i forgot to update proguard :D in this build. All in app texts are in both languages. Presentations have hardcoded texts and it language depends on creator :) we have multilingual presentations on roadmap :)

[felislynx-silae]

@bohuang3d New version is sent. as update. @liaxim framework built upon your branch works perfectly. Sorry for long delay but we had to buy new goggles :D. I've checked 2 presentations before uploading to oculus store of course before sending to review our testers will do full release check.

[thomasflynn]

@felislynx-silae , just checking in...how is it going?

[felislynx-silae]

@thomasflynn we are waiting 4 days for review from oculus, so fingers crossed

[felislynx-silae]

@thomasflynn @liaxim We finally passed technical review. Now only store assets review :) Thank you all for all your help! I'll post link to app as soon as it will be available on store :)

[liaxim]

@felislynx-silae Awesome. Can we close now?

[felislynx-silae]

Yes, i think so, thank you very much!