search

webarkit / ARnft

A small javascript library for WebAR with NFT
GNU Lesser General Public License v3.0
219 stars 53 forks source link

feature : add iframe to webglrenderer #160

Open YMZnew opened 3 years ago

YMZnew commented 3 years ago

I am working on adding iframe to webglrenderer by using css3dobject and css3drenderer but still not working . do you have any suggestion or guideness that can help me developing this feature for augmenting iframes ?

AFAIK we can not add iFrame into a canvas element . so i tried adding iframe into css3dObject and using css3dRenderer .

YMZnew commented 3 years ago

another way i was thinking about is that i can use getTransformationMatrix to get the transMat and display iframe by css webkit-transform with high z-index above the camera canvas . what do you think about it ?

kalwalt commented 3 years ago

Hi @YMZnew i never tried this, so i think i can not help so much. Have you some code somewhere that we can look at? Maybe someone else already did and can help you.

YMZnew commented 3 years ago

Hi @kalwalt . I reviewed your recent publish version and see that you haved removed the dependency of your framework to three.js . that is so cool . now i can use css3Transform to project iframes to marker location. now i need the interpolated matrix which you have calculate on ARnftThreejs.js . is it possible to access its non-obfuscated source ? then i can use css3 to render iframe the same way you have did for threejsRenderer .

kalwalt commented 3 years ago

Yes, now the rendering is in a separate repository https://github.com/webarkit/ARnft-threejs the not bundled code is in the src folder.

kalwalt commented 3 years ago

Unfortunately i have not time to dedicate to this, but post here if you find a solution.

YMZnew commented 3 years ago

what i did is using interpolate matrix to transform iframe with css "matrix3d" style . but the problem is x and y coordinates of detected nft-marker . i listened for "getNftData" event to access the init x,y position of marker , but after that , inside interpolated matrix , i think x and y are not the absolute values which i need . 12th and 13th index of interpolated matrix are the x and y coordinates which move the iframe to ( 0,0 ) point and as i said i think they are not absolute values . do you have any guidance for me ?

YMZnew commented 3 years ago

I'm going to do something tricky to get coordinates of marker in px s , I will test and tell you the result . I think with casting the root cooradinates from local to world and project it using camera in every frames I can locate the iframe coordinates

YMZnew commented 3 years ago

I did what i said above , the coordinates were solved but now i figured out perhaps the interpolated matrix indexes are differ to matrix3d of css style . because in the result , the rotations of iframe are not match with scene ( sometimes is inversed ). i think i should read more about matrix3d of css

kalwalt commented 3 years ago

I did what i said above , the coordinates were solved but now i figured out perhaps the interpolated matrix indexes are differ to matrix3d of css style . because in the result , the rotations of iframe are not match with scene ( sometimes is inversed ). i think i should read more about matrix3d of css

I think they are using a different coord system. Probably but i'm not sure the system origin Is reversed?