webarkit / ARnft

A small javascript library for WebAR with NFT
GNU Lesser General Public License v3.0
219 stars 53 forks source link

More getters and other changes to NFTWorker class #223

Closed kalwalt closed 2 years ago

kalwalt commented 2 years ago

NFTWorker improvements

I added some new getters to the NFTWorker class:

I have removed some not used code and added a worker.onerror listening for Error from Worker. ~Added private update in the ARnft class, that send imageData when it reach some fps.~ this was removed because causing an issue. Renamed ARUtils to ARnftUtils class with also a new function getWindowSize used internally the NFTWorker class, but maybe can be used in other part of the code. getEventTarget also in the ARnft class.

kalwalt commented 2 years ago

if you log the getEventTarget you will see the ARnft is a child of Window:

ARnft  0.11.0
ARnft.ts:163 ARnft init() start...
arNFT_example.html:32
Window {window: Window, self: Window, document: document, name: '', location: Location, …}
ARnft: {ARnft: ƒ}
ARnftThreejs: {SceneRendererTJS: ƒ, NFTaddTJS: ƒ}
THREE: {ACESFilmicToneMapping: 4, AddEquation: 100, AddOperation: 2, AdditiveAnimationBlendMode: 2501, AdditiveBlending: 2, …}
alert: ƒ alert()
arguments: (...)
caller: (...)
length: 0
name: "alert"
[[Prototype]]: ƒ ()
[[Scopes]]: Scopes[0]
atob: ƒ atob()
blur: ƒ blur()
btoa: ƒ btoa()
caches: CacheStorage {}
cancelAnimationFrame: ƒ cancelAnimationFrame()
cancelIdleCallback: ƒ cancelIdleCallback()
captureEvents: ƒ captureEvents()
captureOffScreen: false
captureOnLoad: false
chrome: {loadTimes: ƒ, csi: ƒ}
clearInterval: ƒ clearInterval()
clearTimeout: ƒ clearTimeout()
clientInformation: Navigator {vendorSub: '', productSub: '20030107', vendor: 'Google Inc.', canShare: ƒ, share: ƒ, …}
close: ƒ close()
closed: false
confirm: ƒ confirm()
cookieStore: CookieStore {onchange: null}
createImageBitmap: ƒ createImageBitmap()
crossOriginIsolated: false
crypto: Crypto {subtle: SubtleCrypto}
customElements: CustomElementRegistry {}
defaultStatus: ""
defaultstatus: ""
devicePixelRatio: 1
document: document
external: External {}
fca8217c119444b5ea4ccb77b6b58cd20: fca8217c119444b5ea4ccb77b6b58cd20_constructor {callbacks: {…}, pendingCallbacksUpdate: 0, metadata: null, playbackState: 'none', sendMessage: ƒ, …}
fca8217c119444b5ea4ccb77b6b58cd20_constructor: ƒ ()
fetch: ƒ fetch()
find: ƒ find()
focus: ƒ focus()
frameElement: null
frames: Window {window: Window, self: Window, document: document, name: '', location: Location, …}
getComputedStyle: ƒ getComputedStyle()
getSelection: ƒ getSelection()
history: History {length: 4, scrollRestoration: 'auto', state: null}
indexedDB: IDBFactory {}
innerHeight: 507
innerWidth: 1920
isSecureContext: true
length: 0
localStorage: Storage {debug: '*:error,*:info,*:warn', length: 1}
location: Location {ancestorOrigins: DOMStringList, href: 'http://127.0.0.1:5500/examples/arNFT_example.html', origin: 'http://127.0.0.1:5500', protocol: 'http:', host: '127.0.0.1:5500', …}
locationbar: BarProp {visible: true}
matchMedia: ƒ matchMedia()
menubar: BarProp {visible: true}
moveBy: ƒ moveBy()
moveTo: ƒ moveTo()
name: ""
navigator: Navigator {vendorSub: '', productSub: '20030107', vendor: 'Google Inc.', canShare: ƒ, share: ƒ, …}
albjeremias commented 2 years ago

i was testing this branch.. when i created this issue.. my apologies:

https://github.com/webarkit/ARnft/issues/227 also the examples won't work..

kalwalt commented 2 years ago

i was testing this branch.. when i created this issue.. my apologies:

227 also the examples won't work..

Have you tested the examples in this branch/PR or you are refering to your changes?

albjeremias commented 2 years ago

yes this branch.

kalwalt commented 2 years ago

yes this branch.

I will check again, with which devices did you tested?

albjeremias commented 2 years ago

yes this branch.

I will check again, with which devices did you tested?

firefox, the example of multi:

examples/arNFT_multi_example.html

kalwalt commented 2 years ago

Ok i get the same error with my Oppo A72 with Chrome browser:

ARnft  0.11.0
ARnft.ts:163 ARnft init() start...
arNFT_multi_example.html:63 entities from getEntities static method:  
(3) [{…}, {…}, {…}]
SceneRendererTJS.ts:84 ARnftThreejs version:  0.3.0
arNFT_multi_example.html:107 
(3) ['pinball', 'kuva', 'chalk_multi']
arNFT_multi_example.html:65 [Violation] 'getConfig' handler took 163ms
ARnft.js:1 [Violation] 'getConfig' handler took 178ms
144
[Violation] 'requestAnimationFrame' handler took <N>ms
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 Base path: http://localhost:8080
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 Loading camera at: http://localhost:8080/examples/Data/camera_para.dat
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 No canvas available
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 ARToolkitNFT  0.9.3
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [ARControllerNFT] ARToolkitNFT initialized
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 failed to asynchronously prepare wasm: RangeError: WebAssembly.instantiate(): Out of memory: wasm memory
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 RangeError: WebAssembly.instantiate(): Out of memory: wasm memory
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 Base path: http://localhost:8080
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 Loading camera at: http://localhost:8080/examples/Data/camera_para.dat
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 No canvas available
cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 ARToolkitNFT  0.9.3
cdf140bd-b987-4f46-bffd-1c71fab84736:1 failed to asynchronously prepare wasm: RangeError: WebAssembly.instantiate(): Out of memory: wasm memory

cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1 Uncaught (in promise) RuntimeError: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info.
    at NA (cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1)
    at cff0e3b1-7a1a-4a84-b54e-fac2f1dcc9e0:1
cdf140bd-b987-4f46-bffd-1c71fab84736:1 Base path: http://localhost:8080
cdf140bd-b987-4f46-bffd-1c71fab84736:1 Loading camera at: http://localhost:8080/examples/Data/camera_para.dat
cdf140bd-b987-4f46-bffd-1c71fab84736:1 No canvas available
cdf140bd-b987-4f46-bffd-1c71fab84736:1 ARToolkitNFT  0.9.3
cdf140bd-b987-4f46-bffd-1c71fab84736:1 Uncaught (in promise) RuntimeError: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info.
    at NA (cdf140bd-b987-4f46-bffd-1c71fab84736:1)
    at cdf140bd-b987-4f46-bffd-1c71fab84736:1
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [ARControllerNFT] Camera params loaded with ID 0
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [warning] *** Camera Parameter resized from 640, 480. ***
NFTWorker.ts:109 Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is already detached.
    at G.process (http://localhost:8080/dist/ARnft.js:1:755937)
    at http://localhost:8080/dist/ARnft.js:1:762873
    at Array.forEach (<anonymous>)
    at M._internalUpdate (http://localhost:8080/dist/ARnft.js:1:762855)
    at C (http://localhost:8080/dist/ARnft.js:1:762545)
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Allocated videoFrameSize 307200
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [ARControllerNFT] Got ID from setup 0
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 Loading NFT marker at:  http://localhost:8080/examples/DataNFT/pinball
NFTWorker.ts:109 Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is already detached.
    at G.process (http://localhost:8080/dist/ARnft.js:1:755937)
    at http://localhost:8080/dist/ARnft.js:1:762873
    at Array.forEach (<anonymous>)
    at M._internalUpdate (http://localhost:8080/dist/ARnft.js:1:762855)
    at C (http://localhost:8080/dist/ARnft.js:1:762545)
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Reading /markerNFT_0.fset3
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]   Assigned page no. 0.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]   Done.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Reading /markerNFT_0.fset
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] 
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 ### Surface No.1 ###
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]   Read ImageSet.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Imageset contains 9 images.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]     end.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]   Read FeatureSet.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]     end.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] NFT num. of ImageSet: 9
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] NFT marker width: 893
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] NFT marker height: 1117
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] NFT marker dpi: 120
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info]   Done.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-636
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-648
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-615
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-607
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-590
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-540
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-508
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-437
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] points-269
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Loading of NFT data complete.
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 loadNFTMarker ->  0
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 
{id: 0, width: 893, height: 1117, dpi: 120}
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Page[0]  pre: 31, aft: 31, error = 0.697448
90
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Tracked page 14977968 (max 0).
fdf333af-fdac-4d31-b3d2-3947b229c2fb:1 [info] Tracking lost. -3

it fails to load all the 3 markers, only the first (ony pinball is detected an tracked). Maybe my changes affected the code? Note also that in desktop this not happens.

kalwalt commented 2 years ago

There is also an Out of Memory issue with the Wasm module. I need to check if this happens also with other devices. I wil try to remove the internalUpdate function and see if this issue persist.

kalwalt commented 2 years ago

With my Wiko view instead i don't get the Out of Memory issue. But not all the markers are detected, I think this is to be solved but it's not related to this PR.