hiukim / mind-ar-js

Web Augmented Reality. Image Tracking, Face Tracking. Tensorflow.js
MIT License
2.14k stars 394 forks source link

Features #38 #30 #31: keep objects always visible #327

Open maxfyk opened 1 year ago

maxfyk commented 1 year ago

Features #38 #30 #31: keep objects always visible

Not sure if you need this feature. I wrote it for my project and thought I'd suggest adding it.

hiukim commented 1 year ago

Hi @maxfyk , thanks for your work. I remembered seeing requests like this a couple of times, so definitely something useful!

I'm not sure this needs to be handled in the controller level though. Maybe there's something I miss, but I feel like this can be easily achieved by registering custom event listeners in onTargetFound and onTargetLost. e.g.

in onTargetLost, move the object out from the anchor into the scene directly (and set its pose appropriately), and then in onTargetFound, add the object back to the anchor (and also resume the original pose).

maxfyk commented 1 year ago

Hi @hiukim, I think one might be able to achieve the same functionality this way! Yet, if I understand, you suggest defining custom event listeners that would do the job every time this feature is needed, but then, I think it would be a bypass / a fix, not a feature. What I suggest is adding a quick built-in way to have this functionality.

Also, I implemented it on the controller level for better optimization and to avoid using many workarounds. My understanding of this library is sketchy, so I'm sorry if I missed something!

anselmpaul commented 3 months ago

Hey @maxfyk, I'd like to use your fork since I have the same use case in my project. But I don't understand where in my project I have to pass the parameters stayVisible: true. Here's what I've done:

  1. Cloned your fork
  2. Build the production build and copied the mindar-image-aframe.prod.js file into my project.
  3. Tried passing the parameter like this: <a-scene mindar-image="imageTargetSrc: targets/targets.mind; filterMinCF:0.1; filterBeta: 10; stayVisible: true;" ...
  4. This only logs a warning in the console core:schema:warn Unknown propertystayVisiblefor component/systemmindar-image` and the feature doesn't work.

How do I pass the parameter correctly? Thanks for your help!

anselmpaul commented 3 months ago

Alternatively, I'd like to use your approach @hiukim , but I don't quite understand how to do this

in onTargetLost, move the object out from the anchor into the scene directly (and set its pose appropriately),

I understand how to add the targetLost event listener on a marker, but I don't understand how I would move the object away from the anchor into the scene directly & position it. An example would be highly appreciated 🙏

maxfyk commented 3 months ago

Hey @anselmpaul, I'm glad to hear someone else found this feature helpful! I managed to find this commit that shows the changes I made to the app I was working on at that time to make it use the stayVisible and stayVisibleScale parameters. Could you try modifying your code to match the one that I used?

Screenshot_5

anselmpaul commented 3 months ago

@maxfyk thank you for the quick reply. I found the commit in your repo and replaced the mindar script files with the ones in your repo, that made it work. I also realized why it didn't work: I didn't check out the feature branch before building the mindar js files... Thanks for your help.

Now I wonder if it is possible to toggle this behaviour during runtime? E.g. let the user decide with a button, if the models are kept on screen or not? And to keep the models centered on both axises (currently centered on the x-axis and "standing" on the middle of the y-axis), I would probably need to adapt the world matrix?

maxfyk commented 3 months ago

@anselmpaul

// For toggling the stayVisible, you can try something like this:
function updateStayVisible(value) {
    const sceneEl = document.querySelector('a-scene');
    const arSystem = sceneEl.systems["mindar-image-system"];
    arSystem.stayVisible = value;
}

const updateStayVisibleButton = document.getElementById('updateStayVisibleButton');
updateStayVisibleButton.addEventListener('click', () => {
    updateStayVisible(true);
});

// Though I'm not sure if the parameters refresh during runtime.

// And to lock/center the y-axis too when stayVisible is true, you need to modify the way the "fake" world matrix is calculated:
if (this.stayVisible) {
    const dimensions = this.markerDimensions[i];
    const halfWidth = dimensions[0] / 2;
    const halfHeight = dimensions[1] / 2;
    worldMatrix = [
        1, 0, 0, 0,
        0, 1, 0, 0,
        0, 0, 1, 0,
        -halfWidth, -halfHeight, -(dimensions[0] * dimensions[1]) / (100 + this.stayVisibleScale), 1
    ];
    targetPresent = false;
}

// I haven't tested the code above, so it might not work straight away. Good luck! :)