Add image anchor functionality to the plugin

[CariusLars]

• ideally, users should be able to include the anchor images in two ways: by placing them in the flutter asset folder of their app during compile time or by providing a link to the image to the plugin during runtime
• for inspiration see https://github.com/olexale/arkit_flutter_plugin (iOS) and https://github.com/giandifra/arcore_flutter_plugin (Android) --> the platform-specific implementations can be used as a base to work from and should be wrapped with a common Flutter API

[aleonnet]

Hi @CariusLars I hope your're doing great. Is there any forecast of this feature on roadmap? I'd like to try something like this: https://twitter.com/OlexaLe/status/1174678765592567814

[CariusLars]

Hi @aleonnet , unfortunately I am quite busy at the moment so I won't have time to work on this in the coming weeks. Is this something you might want to look into and contribute? I'd be happy to provide some guidance! Cheers Lars

[aleonnet]

Hi Lars, Sure, I can try if you show me where to start and what should be done! Cheers

On Tue, Nov 2, 2021, 06:28 Lars Carius @.***> wrote:

Hi @aleonnet https://github.com/aleonnet , unfortunately I am quite busy at the moment so I won't have time to work on this in the coming weeks. Is this something you might want to look into and contribute? I'd be happy to provide some guidance! Cheers Lars

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CariusLars/ar_flutter_plugin/issues/17#issuecomment-957261404, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5JYVZJVCHPJ3WJLA3AB7DUJ6VLLANCNFSM47YAFIKQ .

[CariusLars]

Hey @aleonnet & sorry for the late answer! Here's some tips on how you could start:

• Have a look how the ARCore Flutter plugin (link to the example screen using this feature and the ARKit Flutter plugin [link to the example screen using this feature[(https://github.com/olexale/arkit_flutter_plugin/blob/master/example/lib/image_detection_page.dart) handle this feature
• Try to unify the approaches by pulling as much functionality as possible to the Flutter side, here is how a workflow could look:
  • use the SessionManager to allow users to start a Image anchor session and let them provide the images to be tracked from the Flutter side (the session manager should then internally set up iOS and Android and pass the images through)
  • use callbacks in the Session Manager to handle events like images detected. The easiest to start with is just passing back the pose of the image center, users can then decide what to do, e.g. adding a node there
  • for more advanced applications, introduce ImageAnchor as an anchor type and allow interactions using the Anchor Manager, e.g. getting the current tracking state or start/stop tracking of specific images. For this, you can probably just subclass the ARAnchor and handle it with some bookkeeping on the platform sides, similar to how plane anchors work in the plugin now

[aleonnet]

Hi @CariusLars no problem! I've been quite busy too. I checked out the approaches and IOS implementation is more performant, do you know the possible reasons? I've attached the videos to illustrate.

One first doubt, why have you choose to separate the managers instead of keeping them on a unified controller with handlers? Sorry if it seems too basic/obvious. ar_flutter_plugin https://github.com/CariusLars/ar_flutter_plugin/tree/03c1b3aa54ba9de100144ee1a44eeb1d051787f4 /lib https://github.com/CariusLars/ar_flutter_plugin/tree/03c1b3aa54ba9de100144ee1a44eeb1d051787f4/lib /widgets https://github.com/CariusLars/ar_flutter_plugin/tree/03c1b3aa54ba9de100144ee1a44eeb1d051787f4/lib/widgets /ar_view.dart typedef ARViewCreatedCallback = void Function( ARSessionManager arSessionManager, ARObjectManager arObjectManager, ARAnchorManager arAnchorManager, ARLocationManager arLocationManager); ... arkit_flutter_plugin https://github.com/olexale/arkit_flutter_plugin/tree/4fe4aaaf72d8c8a27236fece1227eda8fa731167 /lib https://github.com/olexale/arkit_flutter_plugin/tree/4fe4aaaf72d8c8a27236fece1227eda8fa731167/lib /src https://github.com/olexale/arkit_flutter_plugin/tree/4fe4aaaf72d8c8a27236fece1227eda8fa731167/lib/src /widget https://github.com/olexale/arkit_flutter_plugin/tree/4fe4aaaf72d8c8a27236fece1227eda8fa731167/lib/src/widget /arkit_scene_view.dart typedef ARKitPluginCreatedCallback = void Function(ARKitController controller); typedef StringResultHandler = void Function(String? text); typedef AnchorEventHandler = void Function(ARKitAnchor anchor); typedef ARKitTapResultHandler = void Function(List nodes); typedef ARKitHitResultHandler = void Function(List hits); typedef ARKitPanResultHandler = void Function(List pans); typedef ARKitRotationResultHandler = void Function( List pans); typedef ARKitPinchGestureHandler = void Function( List pinch); ... arcore_flutter_plugin https://github.com/giandifra/arcore_flutter_plugin/tree/d13f151fd71c104a4593d94d9d055aa81b4239c8 /lib https://github.com/giandifra/arcore_flutter_plugin/tree/d13f151fd71c104a4593d94d9d055aa81b4239c8/lib /src https://github.com/giandifra/arcore_flutter_plugin/tree/d13f151fd71c104a4593d94d9d055aa81b4239c8/lib/src /arcore_view.dart typedef void ArCoreViewCreatedCallback(ArCoreController controller);

enum ArCoreViewType { AUGMENTEDFACE, STANDARDVIEW, AUGMENTEDIMAGES }

class ArCoreView extends StatefulWidget { final ArCoreViewCreatedCallback onArCoreViewCreated;

// final UnsupportedHandler onArCoreUnsupported;

final bool enableTapRecognizer; final bool enablePlaneRenderer; final bool enableUpdateListener; final bool debug; final ArCoreViewType type; ...

On Mon, Nov 8, 2021 at 1:50 PM Lars Carius @.***> wrote:

Hey @aleonnet https://github.com/aleonnet & sorry for the late answer! Here's some tips on how you could start:

• Have a look how the ARCore Flutter plugin (link to the example screen using this feature https://github.com/giandifra/arcore_flutter_plugin/blob/master/example/lib/screens/augmented_images.dart and the ARKit Flutter plugin [link to the example screen using this feature[( https://github.com/olexale/arkit_flutter_plugin/blob/master/example/lib/image_detection_page.dart) handle this feature
• Try to unify the approaches by pulling as much functionality as possible to the Flutter side, here is how a workflow could look:
  • use the SessionManager to allow users to start a Image anchor session and let them provide the images to be tracked from the Flutter side (the session manager should then internally set up iOS and Android and pass the images through)
    • use callbacks in the Session Manager to handle events like images detected. The easiest to start with is just passing back the pose of the image center, users can then decide what to do, e.g. adding a node there
    • for more advanced applications, introduce ImageAnchor as an anchor type and allow interactions using the Anchor Manager, e.g. getting the current tracking state or start/stop tracking of specific images

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CariusLars/ar_flutter_plugin/issues/17#issuecomment-963357217, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5JYV4H7N4W56UZTMNFDW3UK75T5ANCNFSM47YAFIKQ .

[CariusLars]

Hi @aleonnet , without knowing the exact implementations, I'm not surprised that the iOS version is more performant. From my experience, ARKit is a LOT more performant than ARCore, and this fact is usually reinforced by the fact that iPhones/iPads are usually more powerful and optimized than their Android counterparts. As for the managers: I've separated them to provide a more modular approach and make it clear which part of the plugin is responsible for each functionality, I found it easier to understand and work with than having everything in one huge controller.

Cheers Lars

[Nico205]

Hey, thanks for both of your efforts! :) Is there any progress on image anchors yet?

Thanks and Friendly Regards Nico

[aleonnet]

Hi Nico Unfortunately I wasn't able code some ideas, due to professional commitments until late March and lack of time except for this and family :) I'm sorry. Regards,

On Sun, Feb 20, 2022, 20:07 Nico @.***> wrote:

Hey, thanks for both of your efforts! :) Is there any progress on image anchors yet?

Thanks and Friendly Regards Nico

— Reply to this email directly, view it on GitHub https://github.com/CariusLars/ar_flutter_plugin/issues/17#issuecomment-1046341359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5JYVYZPEQCIGBGX7PQBCDU4FX3HANCNFSM47YAFIKQ . You are receiving this because you were mentioned.Message ID: @.***>

[Nico205]

Hey, thanks for the reply! Okay, I might take a look at it aswell, since I want to play around with the feature :)

Friendly Regards Nico

[Nico205]

Short progress update: I've implemented the image anchoring for Android as a PoC and it seems to work fine. Going to cleanup my code a bit and start on iOS integration.

Btw: I'm using some GLB models to test and I was unable to see their textures (just black, or only on the inside of the model), is there any known fix for that?

[sithu-aung]

Hello Nico , May I know how to go on with image anchoring. Please kindly share with me if you have an idea or tricks.

[Kurak5]

Hi @Nico205, Could you please share your solution, it will be a great help for me, thanks in advance.

[leeprobert]

@Nico205 - the issue with the black GLB's - I'm seeing the same on iOS but I think it has something to do with how the textures are generated. Open up a new ticket for this and flag me in it, I'll help to investigate this.

[mahmood1996]

Hey, Is there any progress on image anchors yet?

Thanks Mahmoud

[infocodiste]

Hello @Nico205, can you please share your solution for Image Anchoring? you can also help by putting a code snippet here. Thanks.

[utof]

@infocodiste @mahmood1996 since i dont see any comments here #114 i suppose the image anchoring hasn't been implemented yet?

[jyothishreddy18]

Hi @CariusLars

Does this plugin support the Object Pinch to Zoom feature ? Could you please add video streaming in AR. And also gltf files are not rendering from other sources except chicken.gltf why? any solution for this..!!??

Thanks.