Annotation, Billboards and texts extension

jsorel commented 2 years ago

Hello,

I open this issue after the Khronos/OGC meeting (16-12-2021) where we talked about closing the gap between GLTF and Geospatial.

To set the context :

For this case we work on Nautical Charts (maybe you know them as ENC,ECDIS,S-57) Some details on those maps here Between GIS and ECDIS, S-57 and S-52 for marine and military fields

To resume it : S-57+S-52 define map datas and it's symbology, It is the oldest and most achieved map symbology which exist, with over 200 objets types, 500 differents attributes, hundreds of styling rules and close to 600 symbols, plus studied color palettes for Day,Dusk,Night. It's an international standard from IHO and it's used worldwide on boats.

In 2018 we made a test with Cesium 3Dtiles-GLTF-1.0 (before it became an OGC specification) and obtain very good results you can see here : 3D Tiles in action for ENC

Not to much fancy rendering techniques, the correct symbols, clickable, valid for a daily working and professionnal tool. So far so good !

Yet GLTF 2.0 arrived, much easier, PBR, no shader, much better on the paper. But we could not make a decent 3D map anymore. A major downgrade from GLTF 1.0 to 2.0 is the absence of billboards like capability, in 1.0 we could define shaders to achieve billboard visuals. Using 3d meshes to replace them does not work well, not focused, not scaling and uses more resources.

Possible solutions :

adding billboard and texts meshes, like in Cesium CZML -> a bad idea, would be complicated, hard to implement and will end up in being unsupported by engine or have different rendering in the end. Not a good solution for an export/exchange format like GLTF.
Annotation on node transform -> that could work, rather then giving a transform matrix, translation,scale or rotation, let's add a new field :
```
focus : {
target : $node name or id #could be the node where the camera is
constraint : $codes to be defined #to keep the billboard axis align or scaled or whatever
}
```
This way we skip all the troubles from texts and so on, we just provide 'how' the matrices must be combined to obtain the resulting MVP.

Futur problem

combining instancing + billboard ...

Thanks Johann Sorel

jsorel commented 2 years ago

Some comments from Michael.Beale [Autodesk] on the Slack Khronos GLTF. https://app.slack.com/client/T02ACS44ZQA/C02AMPHV479

markupext markupext_0

https://www.youtube.com/watch?v=qz5nSFWBaUw annotations with behavior … annotations with dimensioning …

jsorel commented 2 years ago

One way we could achieve it : like in Blender for constraints :

How to properly Billboard sprites & particles in Blender https://youtu.be/hx8m9w4YyNo?t=49

elalish commented 2 years ago

The difficulty with annotations is you can quickly get into the realm of text reflow, clickable elements, etc. We chose to enable this very generically using the web; any HTML can be easily added to a 3D model and you have the full power of the DOM to back it, all with very little implementation cost for the viewer (examples). The difficulty of course is in portability; HTML is great for any browser-based viewer, but for native requires something like Electron or a WebView.

I would love to see some kind of web-bundle concept for building these experiences in a portable way (think iframe with JS viewer, GLB, HTML annotations, JS for interactions) simply because nearly all the pieces are already standardized, working, and extensible. But I'd be curious to know from the people who want to implement how the web will or will not fit into their product plans.

KhronosGroup / glTF