Implement VulkanSurfaceView for Android platform

[AlessandroBorges]

On Android platform, we must subclass android.view.SurfaceView class in order to draw 3D content. OpenGL ES does it by implementing GLSurfaceView. To implement our VulkanSurfaceView.java, it must be a SurfaceView subclass. Another requirements is to implement binding to Vulkan's WSI. It means:

1. get Surface instance through SurfaceView
2. get the Surface's ANativeWindow* pointer from native side, just like EGL implementation does.
3. Set up VkAndroidSurfaceCreateInfoKHR struct and call Vk10.vkCreateAndroidSurfaceKHR()

GLSurfaceView does the binding this way: (a) Create window surface

   /*
     * Create an EGL surface we can render into.
     */
     GLSurfaceView view = mGLSurfaceViewWeakRef.get();
     if (view != null) {
         mEglSurface = view.mEGLWindowSurfaceFactory.createWindowSurface(mEgl,
                        mEglDisplay, mEglConfig, view.getHolder());
      } else {
        mEglSurface = null;
       }

(b) EGL createWindowSurface select the Surface properly, and pass the object to native side:

 public static EGLSurface eglCreateWindowSurface(EGLDisplay dpy, EGLConfig config, Object win, int[] attrib_list, int offset ){
        Surface sur = null;
        if (win instanceof SurfaceView) {
            SurfaceView surfaceView = (SurfaceView)win;
            sur = surfaceView.getHolder().getSurface();
        } else if (win instanceof SurfaceHolder) {
            SurfaceHolder holder = (SurfaceHolder)win;
            sur = holder.getSurface();
        } else if (win instanceof Surface) {
            sur = (Surface) win;
        }
        EGLSurface surface;
        if (sur != null) {
            surface = _eglCreateWindowSurface(dpy, config, sur, attrib_list, offset);
        } else if (win instanceof SurfaceTexture) {
            surface = _eglCreateWindowSurfaceTexture(dpy, config, win, attrib_list, offset);
        } else {
            throw new java.lang.UnsupportedOperationException(
                "eglCreateWindowSurface() can only be called with an instance of " +
                "Surface, SurfaceView, SurfaceTexture or SurfaceHolder at the moment, " +
                "this will be fixed later.");
        }
        return surface;
    }

(c) EGL's eglCreateWindowSurface native method is this one:

 // C function EGLSurface eglCreateWindowSurface ( EGLDisplay dpy, EGLConfig config, EGLNativeWindowType win, const EGLint *attrib_list )
 private static native EGLSurface _eglCreateWindowSurface( EGLDisplay dpy, EGLConfig config,  Object win,  int[] attrib_list,  int offset     );

(d) The native implementation of eglCreateWindowSurface get ANativeWindow* pointer from passed Surface:

/// implementation from android_opengl_EGL14.cpp

 /* EGLSurface eglCreateWindowSurface ( EGLDisplay dpy, EGLConfig config, EGLNativeWindowType win, const EGLint *attrib_list ) */
static jobject
android_eglCreateWindowSurface
(JNIEnv *_env, jobject _this, jobject dpy, jobject config, jobject win, jintArray attrib_list_ref, jint offset) {
// removed some codes 
 EGLSurface _returnValue = (EGLSurface) 0;
 EGLDisplay dpy_native = (EGLDisplay) fromEGLHandle(_env, egldisplayGetHandleID, dpy);
 EGLConfig config_native = (EGLConfig) fromEGLHandle(_env, eglconfigGetHandleID, config);
    ////////////////////////////
// removed validation code
/////////////////////////////
   //android::sp<ANativeWindow> window; <- THIS IS ANDROID INTERNAL API    
  // window = android::android_view_Surface_getNativeWindow(_env, win); <- THIS IS ANDROID INTERNAL API     
   // Use below code to get ANativeWindow pointer from your java Surface
   // add include for native_window_jni.h header file
   ANativeWindow* window;
   window = ANativeWindow_fromSurface(_env, win);
   if (window == NULL) 
             goto not_valid_surface;   
   ////////////////////////////
// removed validation code
/////////////////////////////

   _returnValue = eglCreateWindowSurface( (EGLDisplay)dpy_native,
                                          (EGLConfig)config_native,
                                           (EGLNativeWindowType)window.get(),
                                            (EGLint *)attrib_list
   );
   // removed some code
  return toEGLHandle(_env, eglsurfaceClass, eglsurfaceConstructor, _returnValue);
}

This way we have essential info to implement VulkanSurfaceView on Android. Some JNI code is necessary to call android::android_view_Surface_getNativeWindow(_env, win); ANativeWindow_fromSurface(_env, win); and get ANativeWindow pointer. Must use this include: #include <android/native_window_jni.h>

On Bor_Vulkan we wrap the pointer with ANativeWindow.java handle and then setup VkAndroidSurfaceCreateInfoKHR.java struct and call VK10.vkCreateAndroidSurfaceKHR()

Those JNI codes must be at VK10.java, using JNIgen.

[AlessandroBorges]

WSI interface may placed in Vulkan.java.

[lbknxy]

I have a look at GLSurfaceView and your VulkanSurfaceView , I confused that there is just one rendering thread in GLSurfaceView or VulkanSurafceView ,but why it needs wait() and notify() methods ?

Can I get SurfaceHolder from SurfaceView and then pass it to native code to create ANativeWindow , and then to create VkSurfaceKHR instead of creating every java file for every Vulkan handle or struct ?

for example ... `class MyView extends SurfaceView implements SurfaceHolder.Callback{

    // other code....

    @Override
    public void surfaceCreated(SurfaceHolder holder) {

        MyThread  thread = new MyThread(this.getHolder()); 
    }

    // other code....

}

class MyThread extends Thread{

 public MyThread(SurfaceHolder holder){

 }

  @Override
  public void run(){

    passHolderToNative(holder);

    // may be wait before.

    // Loop  draw..
    while(...){

        // darw in native code .

        // other code....
    }

  }

}`

[AlessandroBorges]

Hi ! Thanks for your contribution !

We need a rendering thread to deal with external events, mostly coming from GUI thread, like pause, closing, screen rotate, screen resize, etc. Those events must wait current frame rendering process finish before we apply it. Imagine the mess caused by a surface resize in the middle of a drawn... In other moments, we must force thread to wait until we finish a resource acquisition/release operation.

About creating SurfaceKHR for drawing, bor.vulkan.VK10.java class implements several methods for this. But my favorite one is vkCreateWindowSurface() - around line 10.359 - , which creates the correct surface for Android, Win32 or Linux.

There are several ways to implement a rendering loop, but I really like the way it is done at GLSurfaceView / VulkanSurfaceView: using an implementation of Rendering interface. This interface define core operations: onSurfaceCreated(); onSurfaceChange(int width, int height) and onDrawFrame();

The GLThread / VulkanThread will call the Rendering methods in proper manner, inside its main loop. See implementation of run() and guardedRun() in GLThread / VulkanThread.

Any comments are welcome.

[lbknxy]

As you said , we need android::android_view_Surface_getNativeWindow(_env, win) to get a ANativeWindow , because this method come from android_runtime/android_view_Surface.h , so I include these header files in my .cpp file : `

include <android_runtime/android_view_Surface.h>

include <android_runtime/AndroidRuntime.h>

`

But the compiler can't find these two header files and android::android_view_Surface_getNativeWindow(_env, win) method.

my cmake file :

cmake_minimum_required(VERSION 3.4.1)

add_library(native-lib  SHARED  src/main/cpp/native-lib.cpp )

target_link_libraries( native-lib  log  android_runtime )

So , I want to konw : how do you use android::android_view_Surface_getNativeWindow(_env, win) in your bor.vulkan.Vk10.cpp ? Thanks.

[AlessandroBorges]

Sorry for this issue. That code snippet example were draft considering internal NDK stuff. But today I realise that developers should use ANativeWindowAPI instead.

Just replace those includes by #include <android/native_window_jni.h>

and replace the old call for Surface's native window:

`  ANativeWindow* window = android::android_view_Surface_getNativeWindow(_env, win);`

by this one:

  // ANativeWindow* window =  ANativeWindow_fromSurface(JNIEnv* env, jobject surface); 
     ANativeWindow*  window = ANativeWindow_fromSurface( _env, win);

I'll commit those changes to my repo.

[lbknxy]

Now , it works, thanks for your help.