Possible to use my own camera source ? Not stable to have multiple markers?

[kezmanh]

I wonder if it is possible to use my own camera source? I tried to follow arNFT_container_example and expected the video was referring from my own source. However, it changed to ARnft setting one after loading.

Seems ARnft.ARnft.init not support multiple markers as what initWithEntities does?

When I tried to follow arNFT_multi_example, the first marker is always able to be detected with ease. I have three markers. The 2nd sometimes works and the 3rd one always not work.

[kalwalt]

I wonder if it is possible to use my own camera source? I tried to follow arNFT_container_example and expected the video was referring from my own source. However, it changed to ARnft setting one after loading.

No it's not possible at the moment. With the arNFT_container_example is only possible to use a custom "container", that is html code where to inject the rendering page and the loading page, instead of that created by ARnft. I think i will add a custom Camera mode but i haven't a plan for this. I would imrpove before other part of the code.

Seems ARnft.ARnft.init not support multiple markers as what initWithEntities does? No you can do the same. I introduced the initWithEntities function to avoid confusion. Given the https://github.com/webarkit/ARnft/blob/aef6b981db664d30a7fcf6e1bfa72848e5b312f2/examples/arNFT_multi_example.html#L23-L28 WIth the Init function you can:

ARnft.ARnft.init(640, 480, ["examples/DataNFT/pinball", "examples/DataNFT/kuva", "examples/DataNFT/chalk_multi"],  ['pinball', 'kuva', 'chalk_multi'], 'config.json', true)

and this will do the same.

When I tried to follow arNFT_multi_example, the first marker is always able to be detected with ease. I have three markers. The 2nd sometimes works and the 3rd one always not work.

Are you refering to the NFT markers included in the examples or yours? If you are refering to the kuva and chalk_multi NFT markers i think they have not so much DPI and thats the reason. I think i need to provide better markers.

[kezmanh]

@kalwalt Lastly i tried to fix by doing dirty code that skip navigator.mediaDevices code in ARnft.js. It looks very fine when I did the testing. However, when I merged all codes to my existing project, the js sometimes failed to initiate, I wonder if it is related to my dirty fix?

The last console log is ARnftThreejs version: 0.2.0. It should keep loading a lot of resources but it seems didnt start.

It works when I run in desktop browser and a latest model Android. However, it always not run the library when I use not the latest Android and iPhone.

What I do now is create my own video source, then run initWithEntities(). Is it related to memory issue in phone?

navigator.mediaDevices.enumerateDevices()
.then(gotDevices)
.then(function(){
    navigator.mediaDevices.getUserMedia({
          video: constraints
    })
 .then(function(stream){
      window.stream = stream;
      video.srcObject = stream;

       const entities = [
                  { name: 'game1', markerUrl: "/markers/game1" },
                  { name: 'game2', markerUrl: "/markers/game2" },
                  { name: 'game3', markerUrl: "/markers/game3" },
          ];

      ARnft.ARnft.initWithEntities(640, 480, entities,  '/config.json', true)
      .then((nft) => {
      });

[kalwalt]

@kezmanh How you skip the init of the CameraRenderer? Did you https://github.com/webarkit/ARnft/blob/aef6b981db664d30a7fcf6e1bfa72848e5b312f2/src/renderers/CameraViewRenderer.ts#L153 and init your own? I think it would be nice to have the possibility for a custom camera Init but i'm wondering how we can implement?

It works when I run in desktop browser and a latest model Android. However, it always not run the library when I use not the latest Android and iPhone.

Unfortunately i have not so much devices to test, i have no idea for now. Maybe a memory issue? it could be but i can't be sure...

[kalwalt]

The last console log is ARnftThreejs version: 0.2.0. It should keep loading a lot of resources but it seems didnt start.

So can't start ARnftThreejs? can you open an issue on https://github.com/webarkit/ARnft-threejs/issues ? thank you

[kezmanh]

@kezmanh How you skip the init of the CameraRenderer? Did you

https://github.com/webarkit/ARnft/blob/aef6b981db664d30a7fcf6e1bfa72848e5b312f2/src/renderers/CameraViewRenderer.ts#L153

and init your own? I think it would be nice to have the possibility for a custom camera Init but i'm wondering how we can implement?

It works when I run in desktop browser and a latest model Android. However, it always not run the library when I use not the latest Android and iPhone.

Unfortunately i have not so much devices to test, i have no idea for now. Maybe a memory issue? it could be but i can't be sure...

No, I didnt skip this. I simply comment out the following codes only... am i wrong? //if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) { //var g = { audio: !1, video: { facingMode: this._facing, width: { min: 480, max: 640 } } }; //navigator.mediaDevices //.getUserMedia(g) //.then(async (g) => { //this.video.srcObject = g

[kalwalt]

@kezmanh How you skip the init of the CameraRenderer? Did you https://github.com/webarkit/ARnft/blob/aef6b981db664d30a7fcf6e1bfa72848e5b312f2/src/renderers/CameraViewRenderer.ts#L153

and init your own? I think it would be nice to have the possibility for a custom camera Init but i'm wondering how we can implement?

It works when I run in desktop browser and a latest model Android. However, it always not run the library when I use not the latest Android and iPhone.

Unfortunately i have not so much devices to test, i have no idea for now. Maybe a memory issue? it could be but i can't be sure...

No, I didnt skip this. I simply comment out the following codes only... am i wrong? //if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) { //var g = { audio: !1, video: { facingMode: this._facing, width: { min: 480, max: 640 } } }; //navigator.mediaDevices //.getUserMedia(g) //.then(async (g) => { //this.video.srcObject = g

Ok i understood. But in this case How you pass the video stream to the controllers ? see this line: https://github.com/webarkit/ARnft/blob/aef6b981db664d30a7fcf6e1bfa72848e5b312f2/src/ARnft.ts#L171

[kalwalt]

I would modify the ARnft initialize function to pass the video stream into that argument, and i think you will solve the issue. I will make a prototype if i have time.

[kalwalt]

i created a initializeRaw function in the branch raw-initialize. I havent created an example but you can test in this way:

const arnft = new ARnft.ARnft(640, 480, 'config.json');

async function init() {
     // Provide imageData from a canvas see CameraViewRenderer.ts for an example implementation.
     await arnft.initializeRaw(["examples/DataNFT/pinball"], ['pinball'], imageData)
};
init()

[kezmanh]

Thx @kalwalt for the quick fix!

btw i wonder if anyone experienced not stable detection when there are multiple markers? My case now is that i have three markers and users need to scan one by one. It works in iphones even ip6s and some new model Androids. However, a weird case happened in a flagship Android phone in 2018 (I guess not too old enough?) Randomly two of the markers can be detected successfully but the remaining one is always cannot be detected. Uncaught (in promise) RuntimeError: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info. The phone didnt open a lot of apps or many tabs in Chrome. But still out of memory@@ If it is simply a memory issue, is it possible to have a callback to know if marker loaded? Now I can see two "loadNFTMarker -> 0" in console log but not three. I hope there are some callbacks to call so that I can somehow start some new flows for the fix. Thanks.

[kalwalt]

@kezmanh have you tried the code i posted?

btw i wonder if anyone experienced not stable detection when there are multiple markers? My case now is that i have three markers and users need to scan one by one. It works in iphones even ip6s and some new model Androids. However, a weird case happened in a flagship Android phone in 2018 (I guess not too old enough?) Randomly two of the markers can be detected successfully but the remaining one is always cannot be detected. Uncaught (in promise) RuntimeError: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info. The phone didnt open a lot of apps or many tabs in Chrome. But still out of memory@@ If it is simply a memory issue, is it possible to have a callback to know if marker loaded? Now I can see two "loadNFTMarker -> 0" in console log but not three. I hope there are some callbacks to call so that I can somehow start some new flows for the fix. Thanks.

Please open other issues so i can track. This issue was focused to the camera topic. Anyway the lib need other improvements for sure.

[kezmanh]

@kezmanh have you tried the code i posted?

yes but it keeps showing the following error for 5 times, i am investigating if it is my code issue

Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is already detached.
    at NFTWorker.process (https://localhost/js/ARnft.js:240:21)
    at update (https://localhost/js/ARnft.js:144:45)

[kalwalt]

@kezmanh have you tried the code i posted?

yes but it keeps showing the following error for 5 times, i am investigating if it is my code issue

Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is already detached.
    at NFTWorker.process (https://localhost/js/ARnft.js:240:21)
    at update (https://localhost/js/ARnft.js:144:45)

Can you post part of your code or you have a github repository where i can look at?

[kezmanh]

@kezmanh have you tried the code i posted?

yes but it keeps showing the following error for 5 times, i am investigating if it is my code issue

Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is already detached.
    at NFTWorker.process (https://localhost/js/ARnft.js:240:21)
    at update (https://localhost/js/ARnft.js:144:45)

Can you post part of your code or you have a github repository where i can look at?

my code is like this

            navigator.mediaDevices.enumerateDevices()
            .then(gotDevices)
            .then(function(){
                navigator.mediaDevices.getUserMedia({
                   video: constraints
                })
               .then(function(stream){
                 video.srcObject = stream;
                 var canvas = document.getElementById('canvas');
                 var context = canvas.getContext('2d');
                 var img = document.getElementById('video');
                 canvas.width = window.innerWidth;
                 canvas.height = window.innerHeight;
                 context.drawImage(img, 0, 0 );

                requestAnimationFrame(intervalTimer);

                var start = false;

                function intervalTimer() {
                  context.clearRect(0,0,window.innerWidth,window.innerHeight);
                  context.drawImage(video,0,0,window.innerWidth,window.innerHeight);
                  imageData =  context.getImageData(0, 0, window.innerWidth,window.innerHeight);

                  if(!start){
                    start = true;
                    init();
                  }

                  requestAnimationFrame(intervalTimer);
                }

                const arnft = new ARnft.ARnft(640, 480, 'config.json');

                const entities = [
                  { name: 'game1', markerUrl: "markers/game1" },
                  { name: 'game2', markerUrl: "markers/game2" },
                  { name: 'game3', markerUrl: "markers/game3" },
              ];

                async function init() {
                  console.log("start here")
                    await arnft.initializeRaw(["markers/game1","markers/game2","markers/game3"], ['game1','game2','game3'], imageData)
                    .then((nft) => {
                        console.log("loaded ARnft");

[kalwalt]

I think because the worker is receiving the wrong data, it accept only imageData as imageData.data, i will test it.

PostEdit: or maybe better imageData.data.buffer

[kezmanh]

I think because the worker is receiving the worng data, it accept only imageData as imageData.data, i will test it.

PostEdit: or maybe better imageData.data.buffer

I did tried before but seems not work.

this.worker.postMessage({ type: 'process', imagedata: imageData }, [imageData.data.buffer]);

This is from the source code. I think no need to use either .data.buffer or .data for the variable that I passed to the function. I just consoled log imageData and the value looks normal as expected.

[kalwalt]

@kezmanh you are right, you are providing already imageData in the right format. I think it depends on the async/await functions. I m working for a minimal example.

[kalwalt]

I have worked a bit to implement another function initializeRawStream that need a MediaStream instead of ImageData. But i found that the CameraViewRenderer need some more getters and setters see my isuue https://github.com/webarkit/ARnft/issues/216. Before I will improve the CameraViewRenderer and then i will back to this without them it's a bit cumbersome to develop something else.

[kalwalt]

Hi @kezmanh i have worked a bit on this see my PR #245 the example works but i will wait to merge. It will be fine if you can test it. :slightly_smiling_face:

[kalwalt]

Closing this because the feature is merged on master (done with #245)