Environment map

[Methew5]

When I create a reflective material, I see a room in it - default environment map. An example can be seen here Is it possible to change the environment map now?

[ThomasGorisse]

By default, the environment cube map comes from a standard resource image.

It is still not in stable (confirmed full working) state but you can try this

@Override
public void onSessionConfiguration(Session session, Config config) {
    config.setLightEstimationMode(Config.LightEstimationMode.ENVIRONMENTAL_HDR);
    arSceneView.setLightEstimationEnabled(true)
}

With this, the environment will be the ARCore returned one. It is more realistic but less spectacular since the ARCore returned environmental cube map is (since now) quite low resolution (16x16x16x16x16x16)

You can have a look at this discussion for more infos: IndirectLight in an AR context

[blazekv]

@ThomasGorisse I am sorry but I dont get it from your post. Is it possible to change default HDR map? I can see that the default is used for building light probe here . Is there anyway how to change it?

[ThomasGorisse]

You can override in your app the .hdr file here: https://github.com/ThomasGorisse/sceneform-android-sdk/tree/master/core%2Fassets-sources%2Flights%2Fsceneform_default_light_probe Or move the old .hdr lightning system to the last Filament KTLoader using the others environment files which are not used actually but should. It will be great to make a PR on this if you have time. Every contributor is welcome.

[blazekv]

@ThomasGorisse Thank you. Unfortunately this does not work. It looks like this file is actually not used. It is somehow baked into this.

Because if I added new HDR file and change DEFAULT_LIGHTPROBE_ASSET_NAME it ended with this error

2021-07-31 17:53:57.994 10021-10086/? E/Scene: Failed to create the default Light Probe: 
    java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: Light Probe asset "christmas_photo_studio_01_2k" not found in bundle.

If I dont change name and just replace HDR file it does not change reflection image (so I assume that it is not changed and it is loaded from that SFB file).

I am not sure if I can rewrite it to KTXLoader and made pull request but I will try it.

[ThomasGorisse]

You are very probably right about the fact that the environment is inside the sfb. I think I put the original .hdr here because I wanted to keep/save the source file. You can use the 2 other files in the same folder to load them using the Filament KTXLoader. You may probably have to generate them again if the versions are too old.

For the rest of the actual sfb content you can just add a breakpoint at the Filament Indirect Light creation to see what values are used. Then try to find how to pass the same values to the loaded ktxs.

If you can do it, it would be awesome for this Sceneform repo since it's the last SFB usage and so we could remove a lot of unused code.

For exemples of KTX loading, you can have a look at https://github.com/zirman/arcore-filament-example-app and https://github.com/ThomasGorisse/realitycore and directly on the Filament repo https://github.com/google/filament

[blazekv]

@ThomasGorisse Thank you again for your advice. I am struggling with this because I am not Android developer. I cannot find any usage of KTXLoader in that repos you mentiond. Right now I am just exploring LightProbe class and trying to modify it only to load KTX file.

I tried to change loadInBackground method to load IndirectLight instead of LightingDef (if I could manage to do that it will be good starting point to figure out how to rewrite LightingProbe class to use KTX instead of SFB. But I am stuck on that KTXLoader. Maybe I am using it wrong. In this code:

  private CompletableFuture<IndirectLight> loadInBackground(
      Callable<InputStream> inputStreamCreator) {
    return CompletableFuture.supplyAsync(
        () -> {
          if (inputStreamCreator == null) {
            throw new IllegalArgumentException("Invalid source.");
          }

          @Nullable ByteBuffer assetData = null;

          // Open and read the texture file.
          try (InputStream inputStream = inputStreamCreator.call()) {
            assetData = SceneformBufferUtils.readStream(inputStream);
          } catch (Exception e) {
            throw new CompletionException(e);
          }

          if (assetData == null) {
            throw new AssertionError(
                "The Sceneform bundle containing the Light Probe could not be loaded.");
          }

          return KTXLoader.INSTANCE.createIndirectLight(EngineInstance.getEngine().getFilamentEngine(), assetData, new KTXLoader.Options());
        },
        ThreadPools.getThreadPoolExecutor());
  }

On that KTXLoader I always get error:

2021-08-01 11:58:06.721 8971-9021/com.google.ar.sceneform.samples.gltf E/Scene: Failed to create the default Light Probe: 
    java.util.concurrent.CompletionException: java.lang.UnsatisfiedLinkError: No implementation found for long com.google.android.filament.utils.KTXLoader.nCreateIndirectLight(long, java.nio.Buffer, int, boolean) (tried Java_com_google_android_filament_utils_KTXLoader_nCreateIndirectLight and Java_com_google_android_filament_utils_KTXLoader_nCreateIndirectLight__JLjava_nio_Buffer_2IZ)
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:278)
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:284)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1629)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)
     Caused by: java.lang.UnsatisfiedLinkError: No implementation found for long com.google.android.filament.utils.KTXLoader.nCreateIndirectLight(long, java.nio.Buffer, int, boolean) (tried Java_com_google_android_filament_utils_KTXLoader_nCreateIndirectLight and Java_com_google_android_filament_utils_KTXLoader_nCreateIndirectLight__JLjava_nio_Buffer_2IZ)
        at com.google.android.filament.utils.KTXLoader.nCreateIndirectLight(Native Method)
        at com.google.android.filament.utils.KTXLoader.createIndirectLight(KTXLoader.kt:61)
        at com.google.ar.sceneform.rendering.LightProbe.lambda$loadInBackground$1(LightProbe.java:338)
        at com.google.ar.sceneform.rendering.-$$Lambda$LightProbe$EhG1XJmBzKuk42Eeyu6SrJmRNOE.get(Unknown Source:2)

It looks like it is incompatible types directly in com.google.android.filament because nCreateIndirectLight expects Long (as wrapper object for primitive data type long) but Engine returns long as primitive data type. I cannot find any KTXLoader usage so I dont know if I am doing something wrong or if it is really bug inside Filament.

[ThomasGorisse]

You can go with Kotlin if you are more confortable with this language. The plan is to move class by class to Kotlin.

For a recent and accurate sample of KTX loading, the Filament ModelViewer is updated frequently here: https://www.github.com/google/filament/tree/main/android%2Fsamples%2Fsample-gltf-viewer%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fandroid%2Ffilament%2Fgltf%2FMainActivity.kt

[ThomasGorisse]

Environment Lights are now available since this release: Environment Lights + Augmented Faces + Kotlin Begins + Green Flickering Fix + Performances