Status of ARCore post sceneForm abandonment

tellypresence commented 4 years ago

Now that SceneForm has been abandoned*, will ARCore be modified to be easier to adopt for users without a 3D background?

*SceneForm repo now archived (read only); users unable to

raise new issues (or ask questions)
comment on existing issues (or answer questions)
submit PRs
etc

vortice3D commented 4 years ago

Hi Steve:

For me, and of course a lot of for people (like us) using this 3D Engine, this has been a not-very-fair movement from Google.

When they went open-source (1.16), many of us thought it was very good news. We had access to GLTF/GLTB in a complete basis (including animation) by means of gltfio library, and were freed from the "weird" SFA/SFB asset format.

Of course, it had some issues with docs and functionalities not fully polished (compared with 1.15) here and there, but for sure it would be solved in 1.17v and following, we thought.

And then, suddenly, open-source became archived, what seems to be a clear euphemism for abandoned.

From there, I've had some luck moving some issues (not all of them are applicable) to the Filament repo. But it is only a solution when the issue is indeed relative to render aspects.

Best regards.

mpottinger commented 4 years ago

I have pretty much lost faith in ARCore, as you can see here, it does not seem to be a very active community. Issues seem to be not monitored or answered, updates are incremental/small bugfix releases.

I read a rumor that the original ARCore team has fell apart? I'm not sure if that is true, but I wouldn't be suprised.

It seems from now on to be mostly in a kind of 'maintenance' mode instead of active development, and they've pretty much given up on it the way they did Tango and many other projects. There was the ARCore depth API and persistent anchors, but those were announced a long time ago, with no updates on it.

devbridie commented 4 years ago

There are currently no plans to change ARCore's Android SDK to make it easier to adopt for users without a 3D background.

We archived the Sceneform repository to make it clear that we are no longer actively maintaining Sceneform. We open sourced it with the 1.16 release so that you can continue to use it if you find it useful. This means you can also fork the repository to add any additional features that you need.

However, if you are building a new project, consider using the Unity ARCore SDK for Android. It uses Unity to render objects in 3d space and provides an easier way to create an ARCore app.

I want to add that there definitely is active development on the ARCore SDKs. We want to get features like ARCore depth and persistent cloud anchors right before releasing them.

tellypresence commented 4 years ago

I guess the only remaining mystery is, why was SceneForm v1.17.0 added to maven after the SceneForm repo was archived?

tellypresence commented 4 years ago

And finally, google's own @romainguy mentioned in a conference talk last year that unity sort of "takes over" your app, makes it much more difficult to e.g. use AAC "Navigation" component and other modern techniques for app development (since unity apps aren't developed in Android Studio) -- so at least for non-game apps, the advice to base new AR projects on unity is problematic

vortice3D commented 4 years ago

Hi @tellypresence:

Totally right!

Unity is a well-known bloatware factory and, of course, as we say in Spanish, using it for non-game apps is like "Killing flies with guns".

Best regards.

Sergiioh commented 4 years ago

Hello @devbridie, the 'ExternalTexture' used in sceneform doesn't work in the 1.16.0 release. Is there a workaround to make it work?. Best regards.

mpottinger commented 4 years ago

Hi @tellypresence:

Totally right!

Unity is a well-known bloatware factory and, of course, as we say in Spanish, using it for non-game apps is like "Killing flies with guns".

Best regards.

Yeah, I learned that the hard way when I first was getting started out. I didn't know anything about Java programming on Android, or OpenGL, so Unity was the easiest way for me to get started right away with ARCore.

Definitely too bloated for a non-game app. Eventually it got so frustrating that I forced myself to learn Android Studio, and never looked back to Unity. Might be great if you're developing a game, otherwise..... just no.

RGregat commented 4 years ago

Same here, I integrated Arcore/Sceneform with "native" Android UI elements. On my current state of knowledge for Unity this is not possible there. In that case Unity is a no go for me.

netgfx commented 4 years ago

So this means Unity is the only way forward? No way to code things into existence? This seems weird and several steps backwards to be honest. So if I want to develop something with say Flutter I have to know, Dart, Java, some Swift, and now C#? It can't really be that hard in 2020 (although its not the best of years so far) to develop some simple AR app.

vortice3D commented 4 years ago

Hi there:

Trying to tackle the challenge of keeping SceneForm alive for my AR developments, I've the following tasks:

migration to AndroidX;
use of last version of ARCore (1.17.0 at the moment).
use of last version of Filament and gltfio (1.7.0 at the moment).

I'm taking as my starting point the 1.16.0 version, animated GLTF example of the SceneForm repository.

This way I now have my dependencies block in the build.gradle file of sceneform module like this:

dependencies {
    api 'com.google.android.filament:filament-android:1.7.0'
    api 'com.google.android.filament:gltfio-android:1.7.0'
    implementation files("../libs/libsceneform_runtime_schemas.jar")

    api "com.google.ar:core:1.17.0"

    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'com.google.android.material:material:1.1.0'
}

On its part, the build-gradle file of sceneformux is more simple, as it uses the Sceneform SDK built from the source files included in the sceneformsrc folder:

dependencies {
    // Use the Sceneform SDK built from the source files included in the sceneformsrc folder.
    api project(":sceneform")

    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'com.google.android.material:material:1.1.0'
}

Finally, my app module's build-gradle looks like this:

dependencies {
    // Use the Sceneform UX Package built from the source files included in the sceneformux folder.
    api project(":sceneformux")

    implementation 'androidx.appcompat:appcompat:1.0.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    implementation 'com.google.android.gms:play-services-maps:17.0.0'
    implementation 'com.google.android.material:material:1.1.0'
}

Said this, with AndroidX and ARCore 1.17.0 I have not problems, but I can't manage to have SceneForm 1.7.0 working in my app, as it shows the following errors on sceneform/rendering/Renderer.java when run:

cannot find symbol method setClearColor(float,float,float,float)
method beginFrame in class Renderer cannot be applied to given types; required: SwapChain,long; found: SwapChain
cannot find symbol variable targetFrameTimeMilli
cannot find symbol method setClearColor(int,int,int,int)

One after the other, they can be fixed like this:

As view.setClearColor has being deprecated, we must use instead the renderer.setClearOptions API in order to obtain the desired behavior:

public void setClearColor(Color color) {
...
com.google.android.filament.Renderer.ClearOptions options=new com.google.android.filament.Renderer.ClearOptions();
options.clear=true;
options.clearColor=new float[]{color.r, color.g, color.b, color.a};
renderer.setClearOptions(options);
...

Regarding that beginFrame now accepts a v-sync timestamp for accurate frame time measurement (used for frame skipping and dynamic resolution), we need to pass 0 as a second parameter to get the old behavior:
```
public void render(boolean debugEnabled) {
...
if (renderer.beginFrame(swapChainLocal,0)){
    ...
}
}
```

targetFrameTimeMilli is not now a valid parameter for DynamicResolutionOptions, so I comment that line, having then:

public void setDynamicResolutionEnabled(boolean isEnabled) {
...
DynamicResolutionOptions options = new DynamicResolutionOptions();
options.enabled = isEnabled;
//options.targetFrameTimeMilli = 1000.0f / 30.0f;
view.setDynamicResolutionOptions(options);
...
}

For avoid this error I comment the call to setClearColor(), as the strategy used in order to solve the first error is not applicable here:
```
private void initialize() {
...
//emptyView.setClearColor(0, 0, 0, 1);
...
}
```

Said that, when I run my app it crashes, showing the following dump, apparently related to some UX/Fragment issue:

E/ARCore-LocationProvider: Exception during FLP api client construction: com.google.android.gms.location.LocationRequest
    java.lang.ClassNotFoundException: com.google.android.gms.location.LocationRequest
        at java.lang.Class.classForName(Native Method)
        at java.lang.Class.forName(Class.java:453)
        at com.google.ar.core.services.LocationProvider.loadClass(PG:74)
        at com.google.ar.core.services.LocationProvider.<init>(PG:12)
        at com.google.ar.core.Session.nativeCreateSessionAndWrapperWithFeatures(Native Method)
        at com.google.ar.core.Session.<init>(Session.java:14)
        at com.google.ar.sceneform.ux.BaseArFragment.createSessionWithFeatures(BaseArFragment.java:437)
        at com.google.ar.sceneform.ux.BaseArFragment.createSession(BaseArFragment.java:421)
        at com.google.ar.sceneform.ux.BaseArFragment.initializeSession(BaseArFragment.java:391)
        at com.google.ar.sceneform.ux.BaseArFragment.onResume(BaseArFragment.java:350)
        at androidx.fragment.app.Fragment.performResume(Fragment.java:2649)
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:922)
        at androidx.fragment.app.FragmentManagerImpl.moveFragmentToExpectedState(FragmentManagerImpl.java:1238)
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:1303)
        at androidx.fragment.app.FragmentManagerImpl.dispatchStateChange(FragmentManagerImpl.java:2659)
        at androidx.fragment.app.FragmentManagerImpl.dispatchResume(FragmentManagerImpl.java:2625)
        at androidx.fragment.app.FragmentController.dispatchResume(FragmentController.java:268)
        at androidx.fragment.app.FragmentActivity.onResumeFragments(FragmentActivity.java:479)
        at androidx.fragment.app.FragmentActivity.onPostResume(FragmentActivity.java:468)
        at androidx.appcompat.app.AppCompatActivity.onPostResume(AppCompatActivity.java:195)
        at android.app.Activity.performResume(Activity.java:7650)
        at android.app.ActivityThread.performResumeActivity(ActivityThread.java:4017)
        at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:4057)
        at android.app.servertransaction.ResumeActivityItem.execute(ResumeActivityItem.java:51)
        at android.app.servertransaction.TransactionExecutor.executeLifecycleState(TransactionExecutor.java:145)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:70)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1959)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7080)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:965)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.location.LocationRequest" on path: DexPathList[[zip file "/system/framework/org.apache.http.legacy.boot.jar", zip file "/data/app/com.vortice3d.doppelmayrgranada-3aIPrGOQ-iMrv7lx2B5ZVQ==/base.apk"],nativeLibraryDirectories=[/data/app/com.vortice3d.doppelmayrgranada-3aIPrGOQ-iMrv7lx2B5ZVQ==/lib/arm64, /data/app/com.vortice3d.doppelmayrgranada-3aIPrGOQ-iMrv7lx2B5ZVQ==/base.apk!/lib/arm64-v8a, /system/lib64, /vendor/lib64]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:134)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at java.lang.Class.classForName(Native Method) 
        at java.lang.Class.forName(Class.java:453) 
        at com.google.ar.core.services.LocationProvider.loadClass(PG:74) 
        at com.google.ar.core.services.LocationProvider.<init>(PG:12) 
        at com.google.ar.core.Session.nativeCreateSessionAndWrapperWithFeatures(Native Method) 
        at com.google.ar.core.Session.<init>(Session.java:14) 
        at com.google.ar.sceneform.ux.BaseArFragment.createSessionWithFeatures(BaseArFragment.java:437) 
        at com.google.ar.sceneform.ux.BaseArFragment.createSession(BaseArFragment.java:421) 
        at com.google.ar.sceneform.ux.BaseArFragment.initializeSession(BaseArFragment.java:391) 
        at com.google.ar.sceneform.ux.BaseArFragment.onResume(BaseArFragment.java:350) 
        at androidx.fragment.app.Fragment.performResume(Fragment.java:2649) 
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:922) 
        at androidx.fragment.app.FragmentManagerImpl.moveFragmentToExpectedState(FragmentManagerImpl.java:1238) 
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:1303) 
        at androidx.fragment.app.FragmentManagerImpl.dispatchStateChange(FragmentManagerImpl.java:2659) 
        at androidx.fragment.app.FragmentManagerImpl.dispatchResume(FragmentManagerImpl.java:2625) 
        at androidx.fragment.app.FragmentController.dispatchResume(FragmentController.java:268) 
        at androidx.fragment.app.FragmentActivity.onResumeFragments(FragmentActivity.java:479) 
        at androidx.fragment.app.FragmentActivity.onPostResume(FragmentActivity.java:468) 
        at androidx.appcompat.app.AppCompatActivity.onPostResume(AppCompatActivity.java:195) 
        at android.app.Activity.performResume(Activity.java:7650) 
        at android.app.ActivityThread.performResumeActivity(ActivityThread.java:4017) 
        at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:4057) 
        at android.app.servertransaction.ResumeActivityItem.execute(ResumeActivityItem.java:51) 
        at android.app.servertransaction.TransactionExecutor.executeLifecycleState(TransactionExecutor.java:145) 
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:70) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1959) 
        at android.os.Handler.dispatchMessage(Handler.java:106) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.app.ActivityThread.main(ActivityThread.java:7080) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:965)

Any help with this?

Thanks in advance for your time.

ghost commented 4 years ago

Hi @vortice3D

thanks for sharing your efforts for keeping Sceneform alive. I also tried to update Filament but with no success yet.

Did you changed the materials Sceneform uses when updating Filament?

Newer versions of Filament contain breaking changes for which reason it is neccessary to re-compile the materials Sceneform uses. There are a bunch of .matc files in the res/raw/ of sceneformsrc that must be re-compiled using Filaments matc tool.

Sadly, only the compiled materials are available in the Sceneform repository and it is not possible to re-compile them - as mentioned in this issue of the Filament repository. We would need the sources of the materials or try to analyze and rebuild them by ourselfes.

I'm not sure if this is the solution for your issue but it is a neccessary task that is missing on your list anyway.

romainguy commented 4 years ago

Let me ask the Sceneform team if they can share the source of the materials.

romainguy commented 4 years ago

@vortice3D Passing 0 to beginFrame will work but it's better to pass the time supplied by Choreographer instead. It will allow much better frame skipping/dynamic resolution computations on the Filament side.

RGregat commented 4 years ago

@vortice3D thx for sharing you effort to maintain Sceneform a little bit. Maybe we could use your version as the new to go version if someone wants to use Sceneform. And thx to @romainguy to ask the Sceneform team for some more informations regarding the materials.

vortice3D commented 4 years ago

Well, maybe it's time for a "Sceneformium" repository.

ghost commented 4 years ago

Now the Sceneform repository contains all material sources and it's already updated to Filament 1.7.0 in master branch.

The changed Sceneform source code also passes 0 to the beginFrame method:

public void render(boolean debugEnabled) {
    ...
    if (renderer.beginFrame(swapChainLocal, 0)) {
        ...
    }
}

As mentioned by @romainguy this will work but can be optimized by passing the time supplied by the Choreographer.

The render method is called in the doRender of SceneView class. There is also a doFrameNoRepost that calls the doRender and contains a frameTimeNanos:

public void doFrameNoRepost(long frameTimeNanos) {
    ...
    if (onBeginFrame(frameTimeNanos)) {
      doUpdate(frameTimeNanos);
      doRender();
    }
}

This frameTimeNanos is the time when the frame started being rendered and should be the correct time to use in beginFrame. It just needs to be passed down to the render methods.

I will test this and push it to my forked Sceneform repository.

vortice3D commented 4 years ago

Hi @martifier:

First, thank you very much for the work done at your Sceneform repository.

It works fine for me after a couple of tweaks:

adding the following lines to gradle.properties file:


# This project uses AndroidX dependencies.
# The following AndroidX dependencies are detected: androidx.annotation:annotation:1.1.0
android.useAndroidX=true
android.enableJetifier=true


2. adding the following package to the Activity file:

import androidx.appcompat.app.AppCompatActivity;



Best regards.

ghost commented 4 years ago

Thank you, @vortice3D

I did not test the sample app - only my own AR app - yet, so I missed that in the sample app. I will add those tweaks to fix it.

tpsiaki commented 4 years ago

Hi all - We've updated the latest source in the Sceneform github to use Filament 1.7.0. We also added all of the material source files to make it possible to build and run against latest Filament moving forward.

Sergiioh commented 4 years ago

Hi @tpsiaki Thanks for the update!. i'm using the "sceneform_view_renderable.sfb" to show a AugmentedVideo. How can i re-build it to be used in this version?

tpsiaki commented 4 years ago

@Sergiioh Are you using the sfb directly? If so you should be able to copy this snippet of code from ViewRenderable.java to create the same geometry/material setup dynamically: https://github.com/google-ar/sceneform-android-sdk/commit/1548ff3c6ebd4750bf97250e1fea5f61ff6a1c4c#diff-e67af612fdcdf8b9ec3b98dcb4cc149b

If you're just using a ViewRenderable, you should be able to continue using it as you were previously.

Sergiioh commented 4 years ago

@tpsiaki yes, directly as source of a modelRenderable, after that i added an ExternalTexture with the Video.

tpsiaki commented 4 years ago

Got it. Something like this should produce the same result:

Material.builder()
    .setSource(
        context,
        RenderingResources.GetSceneformResource(
            context, RenderingResources.Resource.VIEW_RENDERABLE_MATERIAL))
                .build()
                .thenAccept(
                        material -> {

                          ArrayList<Vertex> vertices = new ArrayList<>();
                          vertices.add(Vertex.builder()
                                  .setPosition(new Vector3(-0.5f, 0.0f, 0.0f))
                                  .setNormal(new Vector3(0.0f, 0.0f, 1.0f))
                                  .setUvCoordinate(new Vertex.UvCoordinate(0.0f, 0.0f))
                                  .build());
                          vertices.add(Vertex.builder()
                                  .setPosition(new Vector3(0.5f, 0.0f, 0.0f))
                                  .setNormal(new Vector3(0.0f, 0.0f, 1.0f))
                                  .setUvCoordinate(new Vertex.UvCoordinate(1.0f, 0.0f))
                                  .build());
                          vertices.add(Vertex.builder()
                                  .setPosition(new Vector3(-0.5f, 1.0f, 0.0f))
                                  .setNormal(new Vector3(0.0f, 0.0f, 1.0f))
                                  .setUvCoordinate(new Vertex.UvCoordinate(0.0f, 1.0f))
                                  .build());
                          vertices.add(Vertex.builder()
                                  .setPosition(new Vector3(0.5f, 1.0f, 0.0f))
                                  .setNormal(new Vector3(0.0f, 0.0f, 1.0f))
                                  .setUvCoordinate(new Vertex.UvCoordinate(1.0f, 1.0f))
                                  .build());
                          ArrayList<Integer> triangleIndices = new ArrayList<>();
                          triangleIndices.add(0);
                          triangleIndices.add(1);
                          triangleIndices.add(2);
                          triangleIndices.add(1);
                          triangleIndices.add(3);
                          triangleIndices.add(2);
                          RenderableDefinition.Submesh submesh =
                                  RenderableDefinition.Submesh.builder().setTriangleIndices(triangleIndices).setMaterial(material).build();
                          ulCorner = ModelRenderable.builder().setSource(
                                  RenderableDefinition.builder()
                                          .setVertices(vertices)
                                          .setSubmeshes(Arrays.asList(submesh))
                                          .build()
                          ).build();
                          // set External Texture in ulCorner.thenAccept
                        }

Sergiioh commented 4 years ago

@tpsiaki Thank you!

SGTjeong commented 4 years ago

Even though I updated sceneform/sceneformux as the github says, I am still getting error

in static filament::MaterialParser filament::details::FMaterial::createParser(backend::Backend, const void , size_t):49 reason: could not parse the material package

What am i missing?

vortice3D commented 4 years ago

Hi @tpsiaki.

First, let me thank you for the effort updating the latest source in the Sceneform github repository to use Filament 1.7.0.

Said this, do you think it could be possible for you to share a comprehensive list of tasks (or methodology) of how to accomplish it, in order that everybody can incorporate future Filament versions at their pace?

About the material source files, what is their location folder in the repository? Also, how must they be build in order to run properly over that future Filament versions?

Best regards.

tpsiaki commented 4 years ago

@vortice3D - I'll look into updating docs to include this info. The short version is to update Filament you need to:

Update the Filament references in the gradle files under sceneformsrc to the new Filament version.
Update Sceneform source corresponding to any API breaking changes that were made in the Filament update.
Replace all of the .matc files in sceneformsrc/sceneform/src/main/res/raw with compiled versions of the corresponding material file from sceneformsrc/sceneform/sampleData

To recompile materials you'll need to download the version of matc for your development platform and run the following command for each material file:

matc --optimize-size --platform=mobile --output sceneformsrc/sceneform/src/main/res/raw/<material_name>.matc sceneformsrc/sceneform/sampleData/<material_name>.mat

tellypresence commented 4 years ago

@tpsiaki Thank you again for the detailed instructions. Material sceneform_view_material shows a warning during conversion:

# In folder sceneform-android-sdk/sceneformsrc/sceneform
$ matc --optimize-size --platform=mobile \
    --output src/main/res/raw/sceneform_view_material.matc \
    sampleData/sceneform_view_material.mat
Ignoring config entry (unknown key): "defines"

Is it safe to ignore? Thank you

tellypresence commented 4 years ago

Also wanted to ask whether there's a modern filament alternative to the legacy sfb light probe file in the same folder as the compiled .matc files

sceneform-android-sdk/\
sceneformsrc/sceneform/src/main/res/raw/sceneform_default_light_probe.sfb

romainguy commented 4 years ago

@tellypresence It's completely safe to ignore. The compiler figures out the defines itself now.

SGTjeong commented 4 years ago

Even though I updated sceneform/sceneformux as the github says, I am still getting error

**in static filament::MaterialParser filament::details::FMaterial::createParser(backend::Backend, const void , size_t):49 reason: could not parse the material package__

What am i missing?

I got it fixed.

I don't know why, but if the project asset directory contains old version's sfb, the app crashes with the error above.

balarayen commented 4 years ago

Even though I updated sceneform/sceneformux as the github says, I am still getting error **in static filament::MaterialParser filament::details::FMaterial::createParser(backend::Backend, const void , size_t):49 reason: could not parse the material package__ What am i missing?

I got it fixed.

I don't know why, but if the project asset directory contains old version's sfb, the app crashes with the error above.

@SGTjeong This error will come if you are using an old matc(compiled with filament 1.4.5 matc).

Either you need to make sure you copied the matc and sfb in sceneform-android-sdk/sceneformsrc/sceneform/src/main/res/raw/*from the latest repository or check if you are having any old matc/sfb file and recompile it with the latest(filament 1.7.0) matc compiler.

vortice3D commented 4 years ago

Hi @romainguy.

Regarding your statement:

Passing 0 to beginFrame will work but it's better to pass the time supplied by Choreographer instead. It will allow much better frame skipping/dynamic resolution computations on the Filament side.

How can SceneView instance be reached from inside Renderer in order to be able to use "Choreographer" returned timestamp in the call to renderer.beginFrame()?

By the way, what exact API must be called?

Best regards.

romainguy commented 4 years ago

Choreographer works with a callback mechanism. When the callback is invoked you get the frame start time as a parameter. Your have to plumb that through Sceneform from wherever the callback is.

ghost commented 4 years ago

@vortice3D SceneView implements the Choreographer FrameCallback. One option is to pass down the frame time as method parameter to the render method following it's call hierarchy. This only needs small adaptions as you can see in this commit

RGregat commented 4 years ago

@vortice3D, @martifier I made all changes to my personal fork. I added sceneform and scenformux as described to the helloscenform sample to see if everything is working, but, I get the following error from Filament

2020-06-17 14:29:41.725 25958-25958/com.google.ar.sceneform.samples.hellosceneform E/Filament: in static filament::MaterialParser *filament::details::FMaterial::createParser(backend::Backend, const void *, size_t):493
    reason: could not parse the material package

I double checked the matc files and recompiled them with the latest CLI Tool. Has someone an idea what the problem could be?

ghost commented 4 years ago

@RGregat The hellosceneform sample is based on Sceneform 1.15.0. In Sceneform 1.15.0 models were integrated using the SFA / SFB formats. There was a Sceneform Plugin (added through the project gradle file) to convert models to SFA / SFB formats on build. The plugin is deprecated since Sceneform 1.16.0 and the usage of .glTF / .glb models is recommended.

Did you convert the Andy model to a .glTF / .glb format, or did you use the old plugin with SFA / SFB formats? I'm not sure if this is compatible with the latest Sceneform.

I added a migrated hellosceneform to my fork with Andy model in .glb format. This works for me without problems.

Hope this helps.

RGregat commented 4 years ago

@martifier Jep, that is working. Because of the exception I thought that the matc files are not correct and because of that I didn't had anything else in mind. Thx! ok cool, now I have a working copy of the latest Sceneform/Filament lib. Is it actually worth to stick to Sceneform for the future or should we migrate directly to Filament? I guess that should be possible.

tellypresence commented 4 years ago

If you are able to integrate filament and ARCore please share details; I have been trying but so far failed https://github.com/google/filament/issues/2646

romainguy commented 4 years ago

The Sceneform source code shows everything you need to do to integrate ARCore and Filament. At some point we'd like to provide a Filament sample that integrates directly with ARCore though.

Sheng-Xuan commented 4 years ago

Thanks to everyone who make sceneform integrated with Filament so that viewRenderer can be used.

RGregat commented 4 years ago

@martifier @vortice3D you already had the chance to peek into the new Depth API? Any idea how to integrate this API to the Sceneform lib?

vortice3D commented 4 years ago

Hi @RGreat;

I just saw the other day that ARCore 1.18 now includes this functionality, but sadly no one of our current devices here at VÓRTICE are compatible with this technology.

Best regards.

ghost commented 4 years ago

@RGregat I didn't have the time to check out the depth API yet. I will look into the API soon but I'm still not sure if I will deep dive into Sceneform to extend it's functionalities or to start building a direct integration of ARCore and Filament for further developments.

Sergiioh commented 4 years ago

Hello everyone, i found a problem while using sceneview. It duplicates the model and has problems rendering the background.

source code: XML <com.google.ar.sceneform.SceneView android:id="@+id/scene_view" android:layout_width="match_parent" android:layout_height="match_parent" android:background="@color/non_ar_gray" /> java code ` mSceneView = (SceneView) findViewById(R.id.scene_view); mScene = mSceneView.getScene();

     andyRenderable = renderable;
    //Setting model for SceneView
    TransformationSystem system = arFragment.getTransformationSystem();
    mSceneNode = new TransformableNode(system);
    mSceneNode.setParent(mScene);
    mSceneNode.setRenderable(andyRenderable);
    mSceneNode.getScaleController().setMinScale(0.01f);
    mSceneNode.getScaleController().setMaxScale(1f);
    mSceneNode.select();

    mSceneNode.setLocalPosition(new Vector3(0f, -0.5f, -1f));
    mScene.addChild(mSceneNode);
    onPlayAnimation(mSceneNode);

` Result: 75519334_193115402120230_1651708826492484094_n

Greetings.

agGitHub commented 4 years ago

Hi everybody. I don't understand where is the new open source Github project for Sceneform. Anybody can share the URL ? Thanks !

RGregat commented 4 years ago

@agGitHub There is none, you have to fork it and maintain it by yourself. Or you use mine (https://github.com/RGregat/sceneform-android-sdk) or from @vortice3D @martifier If theirs is available. I have also an updated demo to test my fork (https://github.com/RGregat/sceneform_demo)

tellypresence commented 4 years ago

Anecdotally, we're stuck on v1.15.0 with fbx to sfb conversion pipeline

Reason is that open sourced v1.16.0 doesn't support model instancing and crashes catastrophically when loading many identical models which is necessary in our scenario.

Believe next filament release may support instancing which could solve the problem (assuming updated filament doesn't cause massive breakage when integrated into v1.16.0 sceneForm) -- just something to be aware of for non-"toy" projects

Originally reported (sample project attached) here: https://github.com/google-ar/sceneform-android-sdk/issues/1046

romainguy commented 4 years ago

Filament APIs already allow you to share textures, vertex and index buffers between multiple renderables. It's not GPU instancing but it would solve your crashes (which are due to memory pressure I assume). Using recent versons of gltfio also allows you to instance gltf assets.

google-ar / arcore-android-sdk

Status of ARCore post sceneForm abandonment #1049