Use dynamic image in app in ARKitMaterialProperty

RobinLbt commented 3 years ago

Hi,

I really like your plugin thank you for all your work !

I would like to know if it's possible to use a dynamic image inside ARKitMaterialProperty (instead of URL or asset image): It's not images I load from the assets, but images I'm building at runtime. I'm doing it using the following code :

 var image = await boundary.toImage(pixelRatio: 1);
    ByteData byteData = await image.toByteData(format: ImageByteFormat.png);
    Uint8List pngBytes = byteData.buffer.asUint8List();
    return pngBytes;

What I'm doing is transforming a rendered widget into a image to display it inside the ARView.

I've done with ARCore, but I'm struggling to do it with ARKit since it doesn't take ByteData nor Uint8List as argument, is there any workaround ?

(I'm not projecting them like your example because they need to be placed at very specific places using LatLng coordinates)

RobinLbt commented 3 years ago

I'm trying a workaround using a generated JSON file containing the encoded image:

    ByteData byteData = await image.toByteData(format: ImageByteFormat.png);
    Uint8List pngBytes = byteData.buffer.asUint8List();
    String img64 = base64Encode(pngBytes);
    Map<String,dynamic> prepareForJson ={"image": img64};
    return json.encode(prepareForJson);

and then use this JSON file for ARKitMaterialProperty:

final material = ARKitMaterial(
 lightningModeName: ARKitLightModel.lambert,
diffuse: ARKitMaterialProperty(image: jsonFile),
);

final plane = ARKitPlane(
materials: [material],
);

final node =ARKitNode(
 geometry: plane,
position : //something,
);

Unfortunately it doesn't work and I get the following error:

Fatal error: Unexpectedly found nil while unwrapping an Optional value: file arkit_plugin/ImageParser.swift, line 11

Looks like the following Swift code cannot parse the image but I have no idea why:


        let data = try Data.init(contentsOf: URL.init(string: name)!)
        return UIImage(data: data)

Any ideas ?

maxTeste commented 3 years ago

@RobinLbt did u find a solution?

RobinLbt commented 3 years ago

@rubenteste no I didn't found any solution yet.

That would be really cool if @olexale could make another argument for the constructor to accept bytes image like ARCore_flutter_plugin