Can I import 3D assets at runtime?

[hcjung2000]

Hi, Thank you for making Sceneform.

Can I import 3D assets like OBJ, FBX and glTF at runtime?

According to this, I can't import 3D assets at runtime.

[pradnya11]

Yes, we would like runtime loading and import from url as well.

[claywilkinson]

You can load .sfb files from URLs by using setSource(Context, Uri) on the ModelRenderable.Builder class.

As far as loading other model types at runtime, this is not supported. Can you explain more about how you would use that capability?

There are 3 main reasons we convert the models at build time:

3D model file formats are very varied in what information they contain in terms of materials and features. Not all these features are supported in Sceneform (for example animation is currently not supported). Also the way materials are defined can vary so it is important to preview what the rendering looks like before using it in the application.

Another reason is 3D models vary is in the scale and orientation. Since we're focusing on AR, we expect models to be "real world" sized. For example a model of a chair would be less than a meter in size. Many models of chairs are much larger when read, so it makes sense to look at the models before using them in an application to make sure the scale is set correctly.

The final reason for the build time conversion approach is performance. File formats of 3D models are structured with editing in mind, not rendering on mobile devices. By converting the models to a uniform structure that works well on mobile devices, both developers and end users have a better experience.

[michalsukupcak]

Hi Clayton,

I'd like to pitch in, too. If load at runtime is out of the question, what would be the recommended approach for an app that's supposed to display a large collection of models, say something like IKEA Place app (https://itunes.apple.com/us/app/ikea-place/id1279244498?mt=8)?

We're working on a similar app for our client that's supposed to display 60+ models (more to come), with multiple different texture variants for each. ARKit allows us to load models at runtime, meaningwe're able to download necessary data on demand (reducing the initial app size) and expand the models by simply adding new obj/mtl files to the backend, without having to issue an update through the store.

We were hoping Sceneform would allow us to do something similar on Android.

[claywilkinson]

@michalsukupcak - Thanks for the question! Yes, Sceneform lets you do the same type of thing. The samples include the models in the .APK to make the samples easy and self-contained. You can load models stored in the cloud without modifying the client application. You just need to convert the assets from whatever format the models were created in to the .sfb format used by Sceneform.

What it involves is creating a gradle project which converts the models to .sfb. It does not have to be the same project as the application.

Then copy the .sfb files to the web server for downloading. Then the application can download the model via URL, or you can download a zip, unzip it on the device and use a file:// URL to access the .sfb file with the ModelRenderable.Builder class.

If you have one model with multiple texture options, you can use the same model and modify the material of the instance by calling Renderable.getMaterial() and use Material.setTexture() to update it. This way you don't need an .sfb for each variant.

[terezo]

Hi @claywilkinson,

I see all the listed reasons for NOT providing models runtime loading as problem to be solved by developers and 3D artists. To set up a proper format, units (to scale correctly), model optimization (maximum polygons etc.). There are formats with rendering in mind (eg. .obj, .glTF), not just for editing as you said.

We are developing IKEA like app too and forcing to install android studio and convert all models (hundreds) by 3D artist or user who upload models on backend, is not acceptable by our client, since it makes the whole process costly and too complicated.

Providing at least .obj runtime loading would help us a lot. Command line script for converting would be an option, though not the best.

[claywilkinson]

Thanks for the feedback @terezo. I'll definitely pass it along. You can build a command line processing pipeline using Gradle (I don't have a sample of that, but you could write a script that iterates over the source models and generates the gradle tasks). Let me know if you have trouble with that and I'll see if can help.

[gdamoreira]

@terezo @claywilkinson Hi, I had a similiar issue, and I'm developing IKEA like app too. After days of research I found a way of converting .obj assets to .sfb assets using only command line, without Android SDK or project. I uploaded this solution on my repository. Here is the link: https://github.com/necrostylery/google-ar-asset-converter

I hope this helps!

Thanks

[terezo]

@claywilkinson did you pass it along? How do you see it with implementation of that .obj runtime loading?

However we start using google-ar-asset-converter, it's workaround which complicate process too much for real production - modelers have to use script and ship both .sfb and .obj and then others have to upload both .sfb and .obj (sufficient for iOS). It duplicates a lot's of data too - we could reuse a textures with .obj, since we have got a lots of variations of the same .obj with just a little differences - e.g. only diffuse texture changes.

[Popeyef5]

@claywilkinson Hi. Thanks for all the previous answers. I (too) am developping an IKEA like app for which I need to be able to display a huge number of objects. I have succesfully converted the 3D models form .obj to .sfb thanks to @necrostylery so I can load them without problems at runtime from an external source. However, I am not able to change the texture of the model at runtime (setTexture does not work). It does work if I create a simple model (for example through ShapeFactory), but not with downloaded files. I was wondering if this is a known bug or if I have to somehow incorporate all the possible textures into the .sfb. Is there a particular way to do this? Thanks in advance!

[manikantagarikipati]

@Popeyef5 @necrostylery , I am developing a IKEA like app and its great to see that you are getting models rendered(.sfb) from remote. is hosting the .sfb in remote and referencing the URL (https://mysite/obj.sfb) using setSource(Context, https://mysite/obj.sfb) started working?? because the sample sfb (generated using the studio sfb generator) internally refers to an .sfm , would want to if the scenform again downloads the nested hyperlinks. or could you share one of the same remote hosted sfb?, it will be really helpful.

[tpsiaki]

In our 1.5 release we added support for loading gltf and glb files directly in our runtime without converting to SFB. See the code snippet here to give it a try: https://developers.google.com/ar/develop/java/sceneform/create-renderables#load_3d_models_at_runtime

[ivstka95]

@hcjung2000 @michalsukupcak @pradnya11 @terezo have found a solution? i have the same problem. appreciate any advice or help!

[kenribs]

well I just wanna know if I could convert .obj, .fbx, etc. to .sfb? through code? :)

[shiaro007-joker]

I'm working on a similar requirement and I'm okay with using gltf format instead of obj. But I'm getting an error when trying to load a gltf from a url at runtime. I'm getting the following exception:

E/ModelRenderable: Unable to load Renderable registryId='{my link}'
    java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: java.io.FileNotFoundException: /data/user/0/{myappname}/cache/1576574330907-0/data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg== (No such file or directory)

Can anyone plz help what is wrong here

[ravsamy]

I'm working on a similar requirement and I'm okay with using gltf format instead of obj. But I'm getting an error when trying to load a gltf from a url at runtime. I'm getting the following exception:

E/ModelRenderable: Unable to load Renderable registryId='{my link}'
    java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: java.io.FileNotFoundException: /data/user/0/{myappname}/cache/1576574330907-0/data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg== (No such file or directory)

Can anyone plz help what is wrong here

I'm working on a similar requirement and I'm okay with using gltf format instead of obj. But I'm getting an error when trying to load a gltf from a url at runtime. I'm getting the following exception:

E/ModelRenderable: Unable to load Renderable registryId='{my link}'
    java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: java.io.FileNotFoundException: /data/user/0/{myappname}/cache/1576574330907-0/data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg== (No such file or directory)

Can anyone plz help what is wrong here

make sure the Url is https:// not http

[dhaval85]

How to create a GLB file using Blender using canonical_face_mesh.fbx?

[gdamoreira]

How to create a GLB file using Blender using canonical_face_mesh.fbx?

Try reading the blender documentation, maybe it helps:

https://docs.blender.org/manual/en/2.80/addons/io_scene_gltf2.html