search

alanross / AlvaAR

World tracking for WebAR. A Javascript library for Augmented Reality to run SLAM in the browser.
GNU General Public License v3.0
349 stars 73 forks source link

Image (marker) tracking? #22

Open skeegan opened 10 months ago

skeegan commented 10 months ago

Hello,

Could AlvaAR be used (or adapted to be used) for recognition and tracking of images (i.e. image markers)?

ThorstenBux commented 10 months ago

Hi mate, can't answer your question but jsartoolkit or mindar are two libs for image tracking on the web.

skeegan commented 10 months ago

thanks @ThorstenBux

They're interesting libraries. Do you know of any others that can enable 3D object tracking in Javascript?

alanross commented 9 months ago

Hi @skeegan,

While AlvaAR includes all the essential subroutines for NFT image tracking as part of its SLAM algorithm, they are primarily designed for SLAM within the library and are not specifically optimized for NFT image tracking. To achieve your goal, you can follow these rough steps:

  1. Feature Detection/Extraction: Use Harris corners to identify and extract unique feature points from a reference image.
  2. Feature Descriptors: Apply ORB descriptors to describe these unique features.
  3. Process Camera Frames: Repeat steps 1 and 2 for each camera frame to detect features and match them with the target image features.
  4. Select Good Matches: Use k-NN (k-nearest neighbors) to filter and keep only the good matches.
  5. Pose and Orientation: Find homography using RANSAC, calculate the corners, and use solvePnP with these corners to obtain rotation and translation vectors.
  6. Camera Projection Matrix: Create and send this matrix to your 3D environment (Three.js, Babylon, AFrame, etc.).
  7. Render and Update: Overlay and dynamically update digital content in your application based on the camera projection matrix.
  8. Continuous Tracking and Updates: Continuously track the target image and update the overlay as needed.

If you check out the library's code, you'll find that all these subroutines are present, though with a focus on SLAM rather than NFT image tracking.

I'm considering adding this feature in the future. In the meantime, you might find these additional resources, complementing what ThorstenBux suggested, helpful: OpenCv, AR.js, martins-js, Object tracking using Homography in OpenCV

skeegan commented 6 months ago

Many thanks @alanross . Your response was very thorough and helpful