search

autodesk-forks / USD

Universal Scene Description
http://www.openusd.org
Other
128 stars 21 forks source link

HdJavaScript and JavaScript API wrappers #2

Open kaischroeder opened 2 years ago

kaischroeder commented 2 years ago

Description of Change(s)

This branch adds several experimental features as proofs of concept to the WebAssembly port of USD. Please see the main WebAssembly branch for build instructions: https://github.com/autodesk-forks/USD/pull/1

This adds:

  1. hdJavaScript (still called hdEmscripten in the code): A C++ RenderDelegate that does some of the heavy lifting of a RenderDelegate to then forward to a simplified RenderDelegate implemented in JavaScript
  2. A THREE.JS RenderDelegate using the above
  3. JavaScript API wrappers for several of the generic functions and classes to work with a USD stage.

After building this branch and running make install, you get a folder hdEmscripten under the build folder that contains all the files needed to create an app based on this. You can for example copy them into the demo project: https://github.com/autodesk-forks/USD/tree/gh-pages/usd_for_web_demos

The only purpose of this PR today is to make the changes easily viewable and to allow people to play with this code. This is far from being in a form that can be merged.

hybridherbst commented 2 years ago

  1. From what I can see skinned meshes (bones / bind poses) are currently not supported. Are there plans to add that? (I'm happy to contribute a couple of test meshes if desired)

  2. Also seems like visibility isn't supported. Here's a test file: VisibleCubeInvisibleSphere.zip. Expected would be that the sphere is hidden while the cube is visible.

  3. There seem to be issues with normals (VisibleCubeVisibleSphere.zip); this file exhibits lots of seams, but only in this web viewer, Omniverse / Unity / QuickLook are fine. image

kaischroeder commented 2 years ago

Many thanks for your bug reports.

  1. From what I can see skinned meshes (bones / bind poses) are currently not supported. Are there plans to add that? (I'm happy to contribute a couple of test meshes if desired)

It doesn't have the highest priority right now for the team I am on, to be honest, because we are focusing more on CAD models :-) I'll add the two other things to our internal bug tracker.

hybridherbst commented 2 years ago

Understood, just want to point out that CAD models to occassionally contain skinned mesh data though :) for cables, wires, soft parts, ... Sometimes also for performance optimization (e.g. one mesh with 1-bone-weights and 300 bone transforms might be faster on some platforms than 300 drawcalls).

kaischroeder commented 2 years ago

Understood, just want to point out that CAD models to occassionally contain skinned mesh data though :) for cables, wires, soft parts, ... Sometimes also for performance optimization (e.g. one mesh with 1-bone-weights and 300 bone transforms might be faster on some platforms than 300 drawcalls).

That's actually pretty clever :-)

hybridherbst commented 2 years ago

That's actually pretty clever :-)

Haha, let me know if you want to hear more production-proven performance optimization tips regarding bringing CAD to realtime 🚲

hybridherbst commented 2 years ago

Is it planned to get this working on mobile? Seems in some cases the SharedArrayBuffer prevents that but in other cases the wasm instantiation never seems to complete. For reference, I added some "status logs" in the top left corner here to show what's happening / what errors might appear: https://usd-viewer.glitch.me/

kaischroeder commented 2 years ago

Is it planned to get this working on mobile? Seems in some cases the SharedArrayBuffer prevents that but in other cases the wasm instantiation never seems to complete. For reference, I added some "status logs" in the top left corner here to show what's happening / what errors might appear: https://usd-viewer.glitch.me/

Maybe it is some timing issue depending on the network or phone. It's working for me in Chrome on Android. I can imagine that the problem is related to the hacky service worker we use to enable cross origin isolation on GitHub Pages - required for SharedArrayBuffer.

It would be great if you could try setting proper http headers on Glitch: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer#security_requirements

hybridherbst commented 2 years ago

@kaischroeder https://usd-viewer.glitch.me/ now sets the headers server-side, I removed the service worker to take that out of the support equation.

(Downside is that it's now not a static page anymore and thus there's a couple seconds spinup delay if the instance happened to go to sleep (after 30mins of inactivity or so))

kaischroeder commented 2 years ago

@hybridherbst I have now found some more time to look into this. the github pages demo indeed stopped working for me as well. your glitch installation also unfortunately does not work for me anymore. I was now able to make it work on mobile with a local https server that sets the right CORS headers.

hybridherbst commented 2 years ago

@kaischroeder mind checking again? I found a glitch (haha) and fixed it. Working with binary files is a bit strange in their editor (they tend to break if you try to view them).

Works for me again at least :)

kaischroeder commented 2 years ago

@hybridherbst Thanks. It works for me on Desktop. :-) On mobile I had to reload once - the first time I tried I got this error message:

Uncaught (in promise) ReferenceError: Usd is not defined at loadUsdFileFromArrayBuffer (VM6:109:7) at HTMLAnchorElement. (VM6:256:13)

hybridherbst commented 2 years ago

Cool, good to know! Might have been a cache thing; mobile devices cache more aggressively than desktop in my experience (you might even get an old file after the first reload in favor of faster loading)