search

Tresjs / tres

Declarative ThreeJS using Vue Components
https://tresjs.org
MIT License
2.27k stars 107 forks source link

MindAR #421

Closed FrankBevr closed 2 months ago

FrankBevr commented 1 year ago

Description

Hola 👋

There is MindAr. MindAr is a threeJS libary for image tracking and other AR Features.

Is it possible to integrate it?

Not too familar with TresJs So far, but maybe with some help.
I can skretch something out to make a tiny AR Expierience.
Does someone with more expierience can take a look and some ideas of how to implent it in a really rough way? ^ ^

Best Regards

Suggested solution

React Three Fiber and mindAR integration

MindAR - Repo

Alternative

No response

Additional context

No response

Validations

FrankBevr commented 1 year ago

Added Note: I just saw XR is on the road map for next year. Yeay 🎉

alvarosabu commented 1 year ago

Yes! I definitely hope to work on XR in general and implement it with the library but honestly have no idea how to start with it. Haha Let's discuss it here

My biggest question @FrankBevr, do I need a headset to test it?

FrankBevr commented 1 year ago

No, you dont need a headset, thats the beauty in my opinion🌻

If you have 10 minutes, test it

  1. gh repo clone hiukim/mind-ar-js
  2. cd mind-ar-js
  3. node --version

    v18.18.2

  4. npm install
  5. npx vite --config vite.config.dev.js
  6. Open a Webbrowser, go to http://localhost:5173
  7. Click on link, which is named Three
  8. accept "allow camera", now you see yourself via your webcam
  9. grap your phone and navigate to https://tinyurl.com/mindar-frank
  10. Hold your phonescreen against your webcam.
  11. 🎉
  12. Go to Example HTML from Hiukim
  13. Spend 3 minutes of reading/looking
  14. Now you know more than before

In General:

The trick with AR is to understand. There is the virtual camera and there is a real camera. You overlay the real camera(webcam) with the virtual camera(ThreeJS Camera). The tricky part is to calucalte the position of the real camera 60 times per second.

Alias "Where is my picture" "There is my picture" "Where is my picture" "There is my picture" "Where is my picture" "There is my picture" 60 times a second. After each frame adjust the virtual camera a bit

More webar libaries:

8th Wall
Zappar
Babylon AR ...

Issue of Intregation

MindAR uses his own thing of ThreeJS Camera.

See this line

The same in blue via react, this line

I don't know yet if thats a actually issue or not, in the react-three-fiber example he just dont use it. See this line , I think he rips the controller from mindAR out and attach it to his own camera somehow, this line But yeah this code is a bit cumbersome to read, especially if i see updateMatrix, yeah sure completly naturally to update Matrix manually 😅🙈

OverAndOut, cheers ✌️

alvarosabu commented 11 months ago

@FrankBevr We started to play around with XR just recently here https://github.com/Tresjs/XR

I see potential as a community package, wdyt?

alvarosabu commented 2 months ago

I'm gonna close due to inactivity. Feel free to reopen if someone wants to work on it. But IHMO should be a community package