Vulkan Support

[mikeskydev]

Hi, I've just found your whole suite of webview plugins and it's looking really promising.

I'm interested in what would be required for Vulkan support, as our app requires Vulkan.

From there, I'd be really interested in retrieving the android surface to use with Unity's upcoming composition layers package https://docs.unity3d.com/Packages/com.unity.xr.compositionlayers@0.5/manual/source-texture-component.html#android-surface for crisp cross platform XR rendering.

From Unity's sample:

package com.unity.xr.compositorlayers;

import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Matrix;
import android.graphics.Paint;
import android.view.Surface;

/**
 * Provides functionality to render a Bitmap onto a Surface
 * received from Unity. Primarily used for handling and displaying
 * texture data passed from Unity to Android native code.
 */
public class TestSurface {

    /**
     * Initializes and renders a Bitmap onto a specified Surface.
     * 
     * @param surfaceObject An object that should be an instance of android.view.Surface.
     * @param bitmap The Bitmap image to be drawn on the surface.
     * @throws RuntimeException if surfaceObject is not an instance of Surface or if bitmap is null.
     */
    public static void InitTestSurface(Object surfaceObject, Bitmap bitmap) {
        if (!(surfaceObject instanceof Surface)) {
            throw new RuntimeException("TestSurface.ctor: supplied object is not an android.view.Surface!");
        }

        Surface surface = (Surface) surfaceObject;
        Canvas canvas = surface.lockCanvas(null);

        if(bitmap != null)
        {
            canvas.drawBitmap(bitmap, new Matrix(), new Paint());
        }
        else
        {
            throw new RuntimeException("Surface or Image is not initialized or is null.");  
        }

        surface.unlockCanvasAndPost(canvas);
    }
}

I don't have much free time but I'd like to voice this out to see if I can help with anything, or attract the right people who might have the requisite knowledge.

[TLabAltoh]

Hi mikeskydev, For Vulkan support, I currently need to implement bind hardwarebuffer to Vulkan's texture, which is managed by unity (This plugin writes webview's surface to hardware buffer using gl + framebuffer object + passthrough shader etc ...).

[TLabAltoh]

Progress

In the latest update, this plugin has added support for Vulkan with certain limitations. The target Android device must support not only Vulkan but also OpenGL ES, as this plugin depends on the GLES API for specific processes.

Additionally, I did not provide sufficient information on how to access the webview's Android surface. Currently, I am utilizing GLRenderer, where the webview's Android surface is managed. However, it is unclear whether this surface can be integrated into the composition layer (I may add API to get this surface from C#).

[firdiar-mita]

@TLabAltoh what is the limitation? in my case the texture is white semi-transparent, the webview texture doesn't seems loaded although the sounds is working. seems like the webview is running, but vulkan texture is not loaded. *im sure calling UpdateFrame every update

Update I also check the logs from AAR, but seems like it's working fine ig got create new SharedTexture(size) and SharedTexture available: True

[TLabAltoh]

@firdiar-mita This plugin depends on some OpenGLES function at native plugin. It mean this plugin will not work on device which only support Vulkan. (but according to this document, in Android, Vulkan needs GLES 3.1 over. Maybe I don't need to consider above ...)

I think OpenGLES are working fine because your attached picture in other issues are rendering webvie frame (is it using OpenGLES don't you?).

I think the problem is caused by the shared texture of the unity side (there is the shared texture of the unity thread and the shared texture of the webview thread that is piped through the hardware buffer). When building the app to use vulkan, the shared texture of the unity side uses the vulkan api (webview side's shared texture uses OpenGLES).

There is a possibility that bind hardware buffer to shared texture in unity thread failed (Binding hardware buffers to each graphics API's native texture is a bit complex, and I only tested the plugin on my own phone and Oculus Quest 2, so I may have missed something).

[TLabAltoh]

From there, I'd be really interested in retrieving the android surface to use with Unity's upcoming composition layers package https://docs.unity3d.com/Packages/com.unity.xr.compositionlayers@0.5/manual/source-texture-component.html#android-surface or crisp cross platform XR rendering.

I moved the composition layer feature issue to here because it is preferable to consider in isolation from the Vulkan API topic.

[TLabAltoh]

I have added a feature that renders webview directly to the composition layers. I’m not sure if you are still interested in this feature since three months have passed since the issue was opened. I have added a comment as a notification.

10 @mikeskydev

[mikeskydev]

Hi, still very much interested, fantastic update!

I am going on paternity leave but will implement this soon after I get back. Thanks again!