React Native - glb/gltf model works in emulator/usb connected device but fails in android apk

[uanama]

So my problem is when I try to run the app on my device using the apk without being connected via USB debugging. I am using android. It works fine on emulators but it doesn't work on apk.

This is the code:

I tried this too, but it never runs updateText("asset creato");.

Its like it gets stuck in: const model = await loader.loadAsync(require('./assets/TempioHera/tempioHeraGLB.glb'));

This also happens with GLTFLoader by three/examples/jsm/loaders/GLTFLoader:

this is the uri i get:

[CodyJasonBennett]

Have you seen useLoader and/or are using the latest version 8.14? expo-asset won't work in release mode, we polyfill behavior for this specifically.

import { useLoader } from '@react-three/fiber'
import { GLTFLoader } from 'three-stdlib'

function Model() {
  const gltf = useLoader(GLTFLoader, require('./path/to/asset.glb'))
  return <primitive object={gltf.scene} />
}

[uanama]

i tried to do this:

this is my package.json, I use @react-three/fiber 8.14:

but i get these errors, both with @react-three/fiber/native and @react-three/fiber

errors from the emulator view:

[lexengineer]

Have you seen useLoader and/or are using the latest version 8.14? expo-asset won't work in release mode, we polyfill behavior for this specifically.

import { useLoader } from '@react-three/fiber'
import { GLTFLoader } from 'three-stdlib'

function Model() {
  const gltf = useLoader(GLTFLoader, require('./path/to/asset.glb'))
  return <primitive object={gltf.scene} />
}

I have similar issue on iOS in the release mode. Could you explain please why expo-asset does not work in the release mode? Or maybe share some resources where we can read about that? Just want to dive into details to make sure I understand.

[CodyJasonBennett]

To clarify, I meant Android APK where uris are shortened for drawables. It remains unknown whether that includes GLB or other assets additionally configured in Metro. I detailed relevant sources in https://github.com/pmndrs/react-three-fiber/pull/2980#issuecomment-1700429450, with fixes rolled up into #2982 of 8.14.

[CodyJasonBennett]

I've confirmed that only image assets are considered drawables, and the polyfills for asset resolution correctly resolve in all cases between dev/APK. This is a new behavior coming from the networking stack and react-native/Expo, so it will be challenging to pinpoint.

[XantreDev]

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

[XantreDev]

In my case i have this error when trying to use useGLTF. I rebuilded native part but it's not the root

 ERROR  The above error occurred in the <ForwardRef> component:

    at anonymous (http://localhost:8081/index.bundle//&platform=android&dev=true&minify=false&app=dev.world.oone.driverapp&modulesOnly=false&runModule=true:469843:24)
    at Suspense
    at Suspense
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)

React will try to recreate this component tree from scratch using the error boundary you provided, ErrorBoundary.
 LOG  createFallbackRerender [Error: Could not load 279: undefined]
 ERROR  Error: Could not load 279: undefined

This error is located at:
    in Unknown
    in FiberProvider
    in CanvasWrapper (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View
    in NativeWind.View
    in Unknown (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View
    in NativeWind.View
    in Unknown (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View (created by AnimatedComponent(View))
    in AnimatedComponent(View)
    in Unknown (created by Moti.View)
    in Moti

[Rakha112]

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null}

[XantreDev]

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null}

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

[Rakha112]

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

oh yes, thank you very much, the texture has been successfully loaded

[CodyJasonBennett]

I'm not sure how "Cannot create URL for blob!" is reachable, looking at https://github.com/facebook/react-native/blob/main/packages/react-native/Libraries/Blob/URL.js#L130. I've just been looking into "Could not load 1: undefined" which is a regression with THREE.FileLoader -- 1 refers to a Metro module reference.

[kdmmanapul]

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

oh yes, thank you very much, the texture has been successfully loaded

Hi Rakha112, I was wondering how did you do it in react-native? care to share your code?

[XantreDev]

He literally shares it))

[kdmmanapul]

He literally shares it))

Where? Sorry I can't seem to find it. because the one in his comments the code their was a quote reply, and not actually his code with the duck.glb

[XantreDev]

Transform duck.glb with https://gltf.pmnd.rs/ you should replace useGLTF with useGLTFCustom from snippets higher. And after it you grab textures images from glb. And use it with useTexture, after it you should replace texture in material

[Rakha112]

hey @kdmmanapul you can use the code given by @XantreGodlike, and as he said you have to load the textures separately by bake the textures into a .png file. You can clone my repo to try it react-native-3D-Example

[kdmmanapul]

Thanks guys @XantreGodlike @Rakha112

[kdmmanapul]

Hmm @Rakha112 @XantreGodlike texture does not seem to work when exported into an APK and run on mobile.

[XantreDev]

Yep, we are downgraded version to earlier one and patching three fiber(

[XantreDev]

@CodyJasonBennett

[CodyJasonBennett]

@kdmmanapul, is this latest R3F? @XantreGodlike, which version are you patching from? I can take a look.

[Rakha112]

Hey @kdmmanapul. I tried it on 3 devices, Redmi Note 10, Samsung Galaxy Tab A8 and Samsung Galaxy Tab S8, all of them can load the Duck and its textures well using @XantreGodlike workaround. I haven't tried using the latest version

Here is the APK file maybe you can try

and this is the version of dependencies that I use

  "dependencies": {
    "@react-three/drei": "^9.82.1",
    "@react-three/fiber": "^8.14.1",
    "assert": "^2.1.0",
    "base64-arraybuffer": "^1.0.2",
    "expo": "^49.0.0",
    "expo-asset-utils": "^3.0.0",
    "expo-file-system": "^15.4.4",
    "expo-gl": "~13.0.1",
    "r3f-native-orbitcontrols": "^1.0.8",
    "react": "18.2.0",
    "react-native": "0.72.4",
    "three": "^0.156.1",
    "three-stdlib": "^2.25.1"
  },

[CodyJasonBennett]

Note that the ^ symbol will install the latest version at the time of install (including minors). If you have a lockfile, that will narrow down which version was resolved. Also when installing a specific version, be sure to pin it by only specifying the exact version (e.g. "8.14.1").

[XantreDev]

Sorry, I think, i should to recheck with latest version, because seems to be i've used old one

[kdmmanapul]

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null}

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

Regarding this one @XantreGodlike @Rakha112 since we added a condition for Array not being supported, how can we add rougness texture on the model?

Having normalMap and RoughnessMap and etc? Since useTexture by default can have an array of maps right?

[CodyJasonBennett]

You can omit that callback. I added unpackAlignment since OpenGL/WebGL defaults have fail cases for data textures -- 4 is the default, 1 is safe for bad data. flipY has no effect but may log a warning if used on older versions of expo-gl.

[uanama]

When I try to use the three-stdlib library in a component I get this error:

This is my package.json:

[CodyJasonBennett]

I'd update three-stdlib or anything before reporting to GitHub. I'll look into it regardless on my end.

[uanama]

I am trying to load a glb that has been compressed either with meshopt or draco, is it possible to load these kind of glb using r3f on both android or iOS? here is how i tried to set the decoders to gltfLoader: (the gltfLoader already loads correctly uncompressed glb models)

but i get these warning messages with this approach:

this is what i get when I try loading a GLB compressed using DRACO:

this is what i get when I try loading a GLB compressed using Meshopt:

[CodyJasonBennett]

I am trying to load a glb that has been compressed either with meshopt or draco, is it possible to load these kind of glb using r3f on both android or iOS?

No, it's not possible to use meshopt or DRACO without JIT support on iOS to implement WebAssembly. Maybe wait until EU 2024 legislature WRT the browser ban which might incidentally help there.

I've merged #3042 for 8.14.6 which reverts changes from #2982 or 8.14 that produces these cryptic promise rejections. I think we'll have to export a native-specific useLoader which uses the above workarounds instead of relying on correct networking behavior upstream.

This undoes changes needed to load textures from a GLB, but should work in Android release mode and anywhere to begin with. Let me know if anything changes on your end and I'll look into a proper fix for textures.