Doesn't work on iOS

[i-panov]

Hello! Under Android everything works, but under iOS only the gray rectangle of the platform in the camera is displayed, but the model is not visible. Maybe there should be some additional specific settings for the iOS platform? Similar to AndroidManifest. I tested this example: https://github.com/CariusLars/ar_flutter_plugin/blob/main/example/lib/examples/localandwebobjectsexample.dart.

[CariusLars]

Hi @i-panov , I just re-compile the example app to my iPad and tested the example you mentioned, it works without problems. Does the app or the debug log in xcode show any errors when you run the example on your iOS device? and could you provide a screenshot of how it looks? Also, which device and iOS version are you compiling to?

[i-panov]

@CariusLars

Task <172E2928-FBBA-4BB6-8E57-7F9E84748D01>.<1> finished with error [-1001] Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={_kCFStreamErrorCodeKey=-2102, NSUnderlyingError=0x281fbc450 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDownloadTask <172E2928-FBBA-4BB6-8E57-7F9E84748D01>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDownloadTask <172E2928-FBBA-4BB6-8E57-7F9E84748D01>.<1>"
), NSLocalizedDescription=The request timed out., NSErrorFailingURLStringKey=https://github.com/KhronosGroup/glTF-Sample-Models/raw/master/2.0/Duck/glTF-Binary/Duck.glb, NSErrorFailingURLKey=https://github.com/KhronosGroup/glTF-Sample-Models/raw/master/2.0/Duck/glTF-Binary/Duck.glb, _kCFStreamErrorDomainKey=4}
makeNodeFromWebGltf received non-200 response code
Async Model Downloading Task completed:  finished
Connection 1: unable to determine interface type without an established connection
Connection 1: unable to determine fallback status without a connection
flutter: [Unable to load renderable [#73786]]

iPhone 7, iOS 14.7.1

[CariusLars]

thanks! this looks like your device (or the app itself) has no internet connection, hence the plugin cannot download the model file. The local object works, right?

[i-panov]

@CariusLars No! The device definitely has access to the Internet!

I also try to download the model (cache) and render it from the device.

import 'package:flutter/material.dart';
import 'package:my_app/utils/ar_helpers.dart';
import 'package:vector_math/vector_math_64.dart';
import 'package:ar_flutter_plugin/managers/ar_location_manager.dart';
import 'package:ar_flutter_plugin/managers/ar_session_manager.dart';
import 'package:ar_flutter_plugin/managers/ar_object_manager.dart';
import 'package:ar_flutter_plugin/managers/ar_anchor_manager.dart';
import 'package:ar_flutter_plugin/ar_flutter_plugin.dart';
import 'package:ar_flutter_plugin/datatypes/config_planedetection.dart';
import 'package:ar_flutter_plugin/datatypes/node_types.dart';
import 'package:ar_flutter_plugin/models/ar_node.dart';
import 'package:ar_flutter_plugin/datatypes/hittest_result_types.dart';
import 'package:ar_flutter_plugin/models/ar_anchor.dart';

class ARPage extends StatefulWidget {
  final String modelUrl;

  ARPage({Key key, @required this.modelUrl}) : super(key: key);

  @override
  createState() => _ARPageState();
}

class _ARPageState extends State<ARPage> {
  ARAnchor _anchor;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('AR просмотр'),
      ),
      body: FutureBuilder(
        future: downloadFile(widget.modelUrl, '/assets/models'),
        builder: (BuildContext context, AsyncSnapshot<String> snapshot) {
          if (snapshot.hasError) {
            return Center(child: Text(snapshot.error.toString()));
          }

          if (!snapshot.hasData) {
            return Center(child: CircularProgressIndicator());
          }

          return ARView(
            planeDetectionConfig: PlaneDetectionConfig.horizontalAndVertical,
            onARViewCreated: (ARSessionManager arSessionManager,
                ARObjectManager arObjectManager,
                ARAnchorManager arAnchorManager,
                ARLocationManager arLocationManager) {
              arSessionManager.onInitialize(
                showFeaturePoints: false,
                showPlanes: true,
                showWorldOrigin: false,
                handleTaps: true,
              );

              arSessionManager.onPlaneOrPointTap = (hits) async {
                final hit = hits.firstWhere((hit) => hit.type == ARHitTestResultType.plane);
                final newAnchor = ARPlaneAnchor(transformation: hit.worldTransform);

                if (await arAnchorManager.addAnchor(newAnchor) == true) {
                  setState(() {
                    if (_anchor != null) {
                      arAnchorManager.removeAnchor(_anchor);
                    }

                    _anchor = newAnchor;

                    final node = ARNode(
                      type: NodeType.webGLB,
                      uri: snapshot.data,
                      scale: Vector3(0.05, 0.05, 0.05),
                      position: Vector3(0.0, 0.0, 0.0),
                      rotation: Vector4(1.0, 0.0, 0.0, 0.0),
                    );

                    arObjectManager.addNode(node, planeAnchor: _anchor as ARPlaneAnchor);
                  });
                }
              };
            },
          );
        },
      ),
    );
  }
}

ar_helpers:

import 'package:http/http.dart' as http;
import 'dart:io';
import 'package:path_provider/path_provider.dart';
import 'dart:convert';

normalizePath(String value) {
  final pathWithStartingSlash = !value.startsWith('/') ? "/$value" : value;

  final pathWithoutLeadingSlash = pathWithStartingSlash[pathWithStartingSlash.length - 1] == '/' ?
  pathWithStartingSlash.substring(0, pathWithStartingSlash.length - 1) :
  pathWithStartingSlash;

  return pathWithoutLeadingSlash;
}

Future<String> downloadFile(String url, String path) async {
  final uri = Uri.parse(url);
  final regexp = new RegExp(r'/([A-Za-z0-9_]+)\.([A-Za-z0-9]+)$');
  final match = regexp.firstMatch(url);
  final filename = match?.group(1) ?? utf8.fuse(base64).encode(url);
  final extension = match?.group(2) ?? '';
  final appDir = await getApplicationDocumentsDirectory();
  final fileDir = new Directory(appDir.path + normalizePath(path));
  await fileDir.create(recursive: true);
  final file = new File("${fileDir.path}/$filename.$extension");

  if (!await file.exists()) {
    final response = await http.get(uri);

    if (response.statusCode == 200) {
      file.writeAsBytes(response.bodyBytes);
    } else {
      throw new Exception(response.body);
    }
  }

  return file.path;
}

Future deleteAppDir(String path) async {
  final appDir = await getApplicationDocumentsDirectory();
  final fileDir = new Directory(appDir.path + normalizePath(path));

  if (await fileDir.exists()) {
    await fileDir.delete(recursive: true);
  }
}

Navigator.push(context, MaterialPageRoute(builder: (builder) => ARPage(modelUrl: 'https://modelviewer.dev/shared-assets/models/Astronaut.glb')));

I get errors:

Task <A3A11EDA-808F-4AFC-B145-B3F990814EE4>.<23> finished with error [-1002] Error Domain=NSURLErrorDomain Code=-1002 "URL не поддерживается" UserInfo={NSLocalizedDescription=URL не поддерживается, NSErrorFailingURLStringKey=/var/mobile/Containers/Data/Application/F05F917E-8BBD-4F5A-BFFC-A2E9C288A2F3/Documents/assets/models/Astronaut.glb, NSErrorFailingURLKey=/var/mobile/Containers/Data/Application/F05F917E-8BBD-4F5A-BFFC-A2E9C288A2F3/Documents/assets/models/Astronaut.glb, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDownloadTask <A3A11EDA-808F-4AFC-B145-B3F990814EE4>.<23>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDownloadTask <A3A11EDA-808F-4AFC-B145-B3F990814EE4>.<23>, NSUnderlyingError=0x2804a9c80 {Error Domain=kCFErrorDomainCFNetwork Code=-1002 "(null)"}}
makeNodeFromWebGltf received non-200 response code
Async Model Downloading Task completed:  finished
flutter: [Unable to load renderable [#3a76e]]

[CariusLars]

This errors because you are using the plugins node type webGLB which tries to download the model from the web, but you are using a local file path.

In the example app of the plugin you mentioned in the initial issue you opened, does displaying the local object work?

If yes, can you check your permission settings and ensure that the app has access to the internet?

[i-panov]

@CariusLars

This errors because you are using the plugins node type webGLB which tries to download the model from the web, but you are using a local file path.

When I try to specify NodeType.localGLTF2 I get an error:

flutter: [Unable to load renderable /var/mobile/Containers/Data/Application/EDA6100D-283A-4C8E-B02C-C801CA056618/Documents/assets/models/Astronaut.glb]

Which once again proves that the model is downloaded, which means that the device has the Internet and it is available to the application!

In the example app of the plugin you mentioned in the initial issue you opened, does displaying the local object work?

Yes

If yes, can you check your permission settings and ensure that the app has access to the internet?

I didn't find such an item in the settings. There is definitely internet on the device!

[CariusLars]

Hi @i-panov,

When I try to specify NodeType.localGLTF2 I get an error:

flutter: [Unable to load renderable /var/mobile/Containers/Data/Application/EDA6100D-283A-4C8E-B02C-C801CA056618/Documents/assets/models/Astronaut.glb]

Which once again proves that the model is downloaded, which means that the device has the Internet and it is available to the application!

The error doesn't necessarily mean that there is a file at that path, it just gives a generic error that loading failed. If the download works, my guess is that it's the wrong file format. If you have a look at the Sceneform Docs, the model loader differentiates between GLTF2 and GLB, so passing NodeType.localGLTF2 but providing a GLB file won't work. Currently, local GLB obejcts are not supported, but that should be really easy to add, just duplicate the model loader function and add localGLB as NodeType. Or simply download a model in GLTF2 format. Haven't you figured out how to load models from the local file system in #27 (would be cool if you could contribute that code in a PR btw)?

I didn't find such an item in the settings. There is definitely internet on the device!

Really out of ideas on that one actually, I'm quite sure it's not a problem with this plugin but rather with the downloading process, so maybe you have to do some more research on the error itself, it seems to be specific to your setup and I can't reproduce the error, sorry!

[i-panov]

@CariusLars Unfortunately, I could not figure out how to compile the project to make a pull request. The autocomplete doesn't work. Also, I don't know Kotlin or Swift.

Suddenly the unexplainable thing happened: everything worked on iOS! But on Android still writes "unable to load renderable" when trying to load a model from the assets.

Also on iOS there is another strange thing: some simple models like that duck from the example are loaded normally, and some are transparent (only the outlines are visible). I'll attach an example of such models.

salade.zip

Such errors also appear on iOS:

scenekit error unsupported conversion short4 float4 scenekit error C3DSourceAccessorCopyDataToAccessor failed

[CariusLars]

Hi @i-panov, so strange it suddenly worked on iOS! So on Android, that error message appears when you compile the sample app or with your own app that uses the plugin? Could you have a look what exact error message the log (e.g. in android studio's logcat) shows?

Also on iOS there is another strange thing: some simple models like that duck from the example are loaded normally, and some are transparent (only the outlines are visible). I'll attach an example of such models.

salade.zip

Such errors also appear on iOS:

scenekit error unsupported conversion short4 float4 scenekit error C3DSourceAccessorCopyDataToAccessor failed

probably the model loading function needs some more work, it seems like some formats or some features within the gltf format are not supported, sorry about this!

@CariusLars Unfortunately, I could not figure out how to compile the project to make a pull request. The autocomplete doesn't work. Also, I don't know Kotlin or Swift.

you have a local clone of the project with your changes though, right?