hiukim / mind-ar-js

Web Augmented Reality. Image Tracking, Face Tracking. Tensorflow.js
MIT License
2.16k stars 399 forks source link

feat: compile the target image just in time (JIT) for AR.js #254

Closed turkyden closed 1 year ago

turkyden commented 1 year ago

Hi, i want to compile the target image Just in time (JIT) but a .mind file in the threejs-mind. #247

Exp:

// Step1: Compile the target image for remote image url
async function loadImage() {
  return new Promise((resolve, reject) => {
    let img = new Image();
    img.crossOrigin = "Anonymous";
    img.onload = () => resolve(img);
    img.onerror = reject;
    img.src = 'https://foo.com/avatar/dengjudeng.png';
  });
}
const images = [];
images.push(await loadImage());
const compiler = new window.MINDAR.IMAGE.Compiler();
const dataList = await compiler.compileImageTargets(images, (progress) => {
  console.log(progress.toFixed(2));
});

+ const exportedBuffer = await compiler.exportData();

// Step2: Render with threejs-mind
const THREE = window.MINDAR.IMAGE.THREE;
const mindarThree = new window.MINDAR.IMAGE.MindARThree({
container: document.querySelector("#container"),
- imageTargetSrc: './assets/card-example/card.mind',
+ imageTargetBuffer: exportedBuffer,
  filterMinCF: 1,
  filterBeta: 10000,
  missTolerance: 0,
  warmupTolerance: 0,
});
const {renderer, scene, camera} = mindarThree;
// ...

Hi, could you help me ? @hiukim

turkyden commented 1 year ago

Or we can compile the remote image online without .mind ? @Parikshit-Hooda @Makio64

hiukim commented 1 year ago

Thanks for the PR. It could be an interesting features to compile images at runtime, but I'm not very convinced it should be done inside the addImageTargets method of the controller. I guess something like this could be achieved by just calling the compiler function to generate the dataList of buffer and then pass that into the controller.