Gaze and View tracking

[arianaa30]

What is the easiest way to get the gaze tracking? Whenever I look at a specific object (that object is visible to me), I want to change its color. Is there such feature inside the API?

[rahul27]

You might want to look at the eye picking sample:

https://github.com/gearvrf/GearVRf-Demos/tree/master/gvr-eyepicking

[arianaa30]

Thanks. As my objects are fixed and are not moving, even simpler, can I play with getRotationPitch() and getRotationYaw? So basically I will locate my view based on my camera's angle, so I can say which object is within my viewport!

[rahul27]

The camera rig has a lookAt vector that you could use that to locate your view:

scene.getMainCameraRig().getLookAt();

The other option is to have a picker set on the camera rig and have a collider attached to your view. The framework would then provide events using IPickEvents whenever you look at your view. Just the way the eye picking sample does it.

[arianaa30]

Great! getLookAt() should be enough for me. One question: I was playing around with the 360 video, and I feel dizzy. I somehow need to make the sphere around me bigger! How can we make the scene get farther? I was trying to make the sphere object bigger using setScale, but that didn't change anything. Then I tried changing the fovY in the configuration file to a higher number (to 110 for instance), but that made the view unstable and weird! Any easy way to do that?

[rahul27]

Using setScale on the sphere would not help. That object is created only to extract the mesh. That mesh is set against the GVRVideoSceneObject. If you want to increase the radius of the sphere - using setScale on the video object should do it:

video.getTransform().setScale(2.0f,2.0f,2.0f);

[arianaa30]

I don't see any difference. Maybe it can not really give the desired output in 360 context?!

[rahul27]

That was what I assumed. You might want to try using a different 360 video.

[arianaa30]

Following the view tracking, I was trying to implement a simple example that whenever I look at a video, I see it, and whenever I don't look at it, it goes off- just to check out how things work.

There are performance issues here; videos flick, they come and go...Is it because this is too computationally complex to be put inside onStep()?

    @Override
    public void onStep() {
        float v[]=scene.getMainCameraRig().getLookAt();
        // each component belongs to [-1,1]
//              Log.d(v[0]+" "+v[1]+" "+v[2], "hello");

        if(v[0]>0 && v[2]>0){
            if(scene.getSceneObjectByName("video1")==null)
                scene.addSceneObject(video1);
            else
                scene.removeSceneObject(video1);    
        } 
        else if(v[0]>0 && v[2]<0){
            if(scene.getSceneObjectByName("video2")==null)
                scene.addSceneObject(video2);
            else
                scene.removeSceneObject(video2);
        }
    }

[rahul27]

There are few things to consider here.

• The call getSceneObjectByName iterates through all the scene objects in your sample. You call this twice. You could instead save the state - added or not. It would avoid the getSceneObjectByName call.
• You could try using the scene.setEnable() call instead of the addSceneObject() call. It is comparatively less expensive.

If you still notice performance issues then I would recommend looking at the Collider and Picker model used in the eye picking sample.

[arianaa30]

I'll check this now. Where is setEnable? I can not find setEnable(), neither on video object, nor on scene.

[rahul27]

The call was added recently https://github.com/Samsung/GearVRf/pull/606.

Are you using a pre-compiled library?

[arianaa30]

Oh, yes. I'm using a binary gvr-exported.jar that Thomas gave me a while ago. I had some problems referencing the Framework project. Is there an updated pre-compiled binary somewhere?

[rahul27]

Not at the moment.

[thomasflynn]

we need to spin another build 'soon'. we should probably see if we can publish 'nightlies' as well.

[arianaa30]

I see. That would be cool! I'll try building a fresh copy of the source code as Mihail suggested to see if I can succeed referencing it!

[arianaa30]

I just reiterated all building process multiple times with my Eclipse, but couldn't succeed. It builds successfully, but when trying to reference it on the sample app, it throws app closed unexpectedly error. Here are my steps. What is missing?? 1- I downloaded a fresh copy of GearVR source from git and just extracted it (GVRf folder) 2- I downloaded oculus mobile SDK (new version is 1.0.3) and put it alongside/ parallel with GVRf folder (remove version so it is ovr_mobile_sdk folder) 3- As suggested, I copied the 3 .jar file into my gvrf's project libs folder (inside GVRf/Framework/framework or whatever). So there is no arm_v7a now. Later after build apparently it is added. Anyways, I get no errors for build. 3- Now I simply import GVRf/Framework project in my eclipse (it is called Framework), clean it, and build. I get no errors for build. So things are good for build. 4- Now I downloaded your demos, and import one of them, say gvr-360photo. It has NO libs folder inside the project as the online document doesn't mention anything about it. 5- I go to build-path of the sample app --> project--> add and select Framework--> OK. All errors inside the project due to missing API goes away. 6- Run! And I get the error:

06-29 16:46:47.049: E/AndroidRuntime(4431): java.lang.RuntimeException: Unable to instantiate activity ComponentInfo{org.gearvrf.gvr360Photo/org.gearvrf.gvr360Photo.Minimal360PhotoActivity}: java.lang.ClassNotFoundException: Didn't find class "org.gearvrf.gvr360Photo.Minimal360PhotoActivity" on path: DexPathList[[zip file "/data/app/org.gearvrf.gvr360Photo-2/base.apk"],nativeLibraryDirectories=[/data/app/org.gearvrf.gvr360Photo-2/lib/arm64, /vendor/lib64, /system/lib64]]
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3094)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3350)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread.access$1100(ActivityThread.java:222)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1795)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.os.Handler.dispatchMessage(Handler.java:102)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.os.Looper.loop(Looper.java:158)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread.main(ActivityThread.java:7229)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at java.lang.reflect.Method.invoke(Native Method)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)
06-29 16:46:47.049: E/AndroidRuntime(4431): Caused by: java.lang.ClassNotFoundException: Didn't find class "org.gearvrf.gvr360Photo.Minimal360PhotoActivity" on path: DexPathList[[zip file "/data/app/org.gearvrf.gvr360Photo-2/base.apk"],nativeLibraryDirectories=[/data/app/org.gearvrf.gvr360Photo-2/lib/arm64, /vendor/lib64, /system/lib64]]
06-29 16:46:47.049: E/AndroidRuntime(4431):     at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.Instrumentation.newActivity(Instrumentation.java:1095)
06-29 16:46:47.049: E/AndroidRuntime(4431):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3084)
06-29 16:46:47.049: E/AndroidRuntime(4431):     ... 9 more
06-29 16:46:47.049: E/AndroidRuntime(4431):     Suppressed: java.lang.NoClassDefFoundError: org.gearvrf.gvr360Photo.Minimal360PhotoActivity
06-29 16:46:47.049: E/AndroidRuntime(4431):         at dalvik.system.DexFile.defineClassNative(Native Method)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at dalvik.system.DexFile.defineClass(DexFile.java:226)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at dalvik.system.DexFile.loadClassBinaryName(DexFile.java:219)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at dalvik.system.DexPathList.findClass(DexPathList.java:338)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:54)
06-29 16:46:47.049: E/AndroidRuntime(4431):         ... 13 more
06-29 16:46:47.049: E/AndroidRuntime(4431):     Suppressed: java.lang.ClassNotFoundException: org.gearvrf.gvr360Photo.Minimal360PhotoActivity
06-29 16:46:47.049: E/AndroidRuntime(4431):         at java.lang.Class.classForName(Native Method)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
06-29 16:46:47.049: E/AndroidRuntime(4431):         at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
06-29 16:46:47.049: E/AndroidRuntime(4431):         ... 12 more
06-29 16:46:47.049: E/AndroidRuntime(4431):     Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available

[liaxim]

It could be 5. Make sure you add the library project in the Android section of the project properties. Do not touch the Java Build Path section.

[arianaa30]

I already changed it by hand by modifying project.properties. Anyways, I did it: Android-->add--> Framework (I remove the pre-written one). Of course I also change the target-SDK (23 for me). \ I have to go to build-path, either add Framework's generated .jar file or add Framework as a project. Otherwise the errors don't go and it doesn't allow me to even run the app.

[liaxim]

Something is really wrong if you have to add to the build-path. There is no way your apk would work like this. Remove the jars from the Java Build Path and let me know what are the errors you get.

[arianaa30]

Regular Import not found error. OK. I remove the Framework from Properties-> build-path->Project. Lots of errors inside the source code. For instance: The import org.gearvrf.GVRActivity cannot be resolved

[liaxim]

Can you post a screenshot of the sample's Project Properties->Android?

[liaxim]

This probably will not help but you might also restart Eclipse.

[arianaa30]

Restarted and no change. For import missing problems it only requires add through library or project. Here they are:

[liaxim]

Please remove through the UI in Project Properties->Android the Framework library. Then add it again via the UI. Absolute paths didn't work for me.

[arianaa30]

OK. I readded it, but the address is automatically given through UI. I have no control. In any case I changed it to relative on the project.properties file: android.library.reference.1=../../../../../Framework/framework/src/main

No chance. It doesn't remove the errors.

[liaxim]

@arianaa30 Do you need more information about the gaze and view tracking?

About the build issues: there is something special about your environment that I cannot infer. You might try creating a new workspace and/or deleting and re-importing the projects. You could also consider using Android Studio too.

[arianaa30]

I have no question at this moment, am trying to implement based on eyePicking. I can ask questions later. For the build, it would be best you make a pre-built .jar like what you have for the older version. All open-source projects they provide source and binary at the same time for each release. I was trying to simply replace the contents of old gvr-exported.jar with new .class and .jar and .so files (.jar is basically a .zip file), but this stupid way didn't work!

[arianaa30]

To be honest I'm starting to LOVE your API; pretty good managed and quick and good support! This building issues is only messing with me.

[rahul27]

Did you follow these steps from the gearvrf website:

• Eclipse IDE If you do not have Eclipse: Download Eclipse IDE for Java Developers from https://eclipse.org/downloads/ Download Android Development Tools (ADT) for Eclipse and configuration instructions from http://developer.android.com/sdk/installing/installing-adt.html Download the NDK plugin with Eclipse CDT (C/C++ Development Tooling) from http://tools.android.com/recent/usingthendkplugin NOTE: Do NOT install a CDT version higher than 8.0.2 or you will encounter parser errors at the JNI level, even though these will not affect building and running your GearVRf application.
• Build Configuration for Eclipse For BOTH methods, locally building AND using a pre-built framework, the following must be properly configured. Properly configure Eclipse: For Android Lollipop, ensure the current Java compiler level is at least 1.7: Click Window > Preference > Java > Compiler > JDK Compliance > Compiler compliance level Verify the compiler level is 1.7 or above. If it is NOT, set it to 1.7. Ensure the JDK version 1.7 or above is in Eclipse: Click Window > Preference > Java > Installed JREs Verify the JDK version is 1.7 or above. If it is NOT, click Add and follow the instructions to put it in. Click Window > Preferences > Android > Build, and UNcheck Force error when external jars contain native libraries If NOT using Eclipse with the ADT bundle, properly configure the Eclipse Memory Settings for Android development NOTE: For GearVRf development, it is recommended to set Xms = 512m and Xmx = 1024m in order to not encounter Java heap size errors. Click Project > UNcheck Build Automatically Click Project > Clean... , checkmark Start a build immediately, and checkmark Build only the selected projects

[arianaa30]

Yes I followed them in the beginning. One concern is regarding the CDT version that says do not use higher than 8.0.2. Is it valid to assume this is not updated?! My eclipse is Mars and apparently each eclipse has it own version of CDT associated with it. So for Mars it was 8.8.1 and I had to download this version. Probably I can not install CDT 8.0.2(for Indigo) on it.

[NolaDonato]

I wrote some documentation on picking that shows how to use events generated by GVRPicker to change the color of an object. http://gearvrf.org/bin/view/GearVRfDeveloperGuide/Picking This is now the easiest way to get gaze tracking.

[arianaa30]

That is great! I'll check it out today.

[arianaa30]

My build issues were resolved and my gaze tracking performance is better now using the new method setEnable(). Thanks for your hints so far. Feel free to close this. I'll ask if I encountered any problems.