The camera may not start up (AR_ERROR_NOT_YET_AVAILABLE)

[swantmon]

I try to render the ArFrame into a render target using OpenGLES 3.2. The ArCore example shows the usage of GLES2 but inside GLES3 the extension is not available. Now I have found the extension GL_OES_EGL_image_external_essl3 to use samplerExternalOES. It seems to be that Unity3D uses the same GLES extension.

My includes and defines:

include "GLES3/gl3.h"

#ifndef GL_OES_EGL_image_external
#define GL_OES_EGL_image_external 1
#define GL_TEXTURE_EXTERNAL_OES           0x8D65
#define GL_TEXTURE_BINDING_EXTERNAL_OES   0x8D67
#define GL_REQUIRED_TEXTURE_IMAGE_UNITS_OES 0x8D68
#endif /* GL_OES_EGL_image_external */

The creation of the texture is similar to the ArCore example:

glGenTextures(1, &g_TextureID);
glBindTexture(GL_TEXTURE_EXTERNAL_OES, g_TextureID);
glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

ArSession_setCameraTextureName(m_pARSession, g_TextureID);

Inside the rendering loop:

glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_EXTERNAL_OES, g_TextureID);

My shaders look like this:

constexpr char kVertexShader[] = R"(
#version  320 es

layout(location = 0) in vec2 in_UV;
layout(location = 0) out vec2 out_UV;

void main()
{
    vec2 Vertices[4];

    Vertices[0] = vec2(-1.0f, -1.0f);
    Vertices[1] = vec2( 1.0f, -1.0f);
    Vertices[2] = vec2(-1.0f,  1.0f);
    Vertices[3] = vec2( 1.0f,  1.0f);

    out_UV = in_UV;

    gl_Position = vec4(Vertices[gl_VertexID], 0.0f, 1.0f);
}
)";

constexpr char kFragmentShader[] = R"(
#version 320 es

#extension GL_OES_EGL_image_external_essl3 : require

precision mediump float;

layout(location = 0) uniform samplerExternalOES in_ExtOESTexture;

layout(location = 0) in vec2 in_UV;

layout(location = 0) out vec4 out_Output;

void main()
{
    out_Output = texture(in_ExtOESTexture, in_UV); //vec4(in_UV, 0, 1);
}
)";

Rendering only the in_UV with out_Output = vec4(in_UV, 0, 1);, the result is a perfectly looking UV texture on the screen. But using the texture, everything is black. The size of the the texture (using textureSize(in_ExtOESTexture, 0)) is zero for both dimensions. ArSession_update(...) is called by the way and the result is AR_SUCCESS. If I try to get image metadata the result is AR_ERROR_NOT_YET_AVAILABLE. ArCore seems to work because the log gives me some information about non-available RANSAC features if I put my finger in front of the camera.

Any idea how to solve this problem?

[swantmon]

Moreover, I have to say that I'm using ARCore within a native activity.

[swantmon]

I have created a little example project extending the provided native activity example. It's a fork with a new branch for the native activity using ArCore. If anyone is interested: https://github.com/SchwandtTobias/android-ndk/tree/native-activity-arcore

[inio]

We're a little confused on this one. At the very least this sounds like a docs bug. Thanks for the repro case - that'll make tracking it down easy.

[swantmon]

No problem. Maybe I have missed something very simple. The permission code is crap, but it works for this example. The first time you start the application ArCore has to fail. So, please test the application for granted permissions. Hope to hear from you very soon. Do you have the same result? I'm using a Pixel 2 XL.

[Xanvial]

Is the texture still showing black after your application is minimized and then maximized again? If it indeed fixes the texture, try to pausing and resuming the AR Session before start drawing.

Btw, do you use AR Core from playstore or from preview from this repo? IIRC it has different version, so maybe it's related with your problem.

[swantmon]

Pausing and resuming doesn't help. I have already tried. But I didn't have tested the ArCore lib from the play store (I have used the one from this repo). As soon as I am in the office I will test it.

[swantmon]

Okay. I found the bug or my mistake. After resetting my android device I forgot to install the "arcore-preview2.apk". The strange thing is that the provided ArCore example 'hello_ar_c' works perfectly and I got no "AR_UNAVAILABLE_ARCORE_NOT_INSTALLED" error at all.

[inio]

Thanks for that update. You're right, that is very odd.