OpenFlight VS glTF - visualization performance

[cnreediii]

Ryan Franz

There are always tradeoffs in creating data geared toward visualization alone. Creating a single batched 3D model is likely better performance, but at a loss of flexibility.

So I think we would need to address the assumptions up front on how this is done. Here are the assumptions that I think would have to be made (for the CDB performance case):

Assuming that the device supports a WGS84 ellipsoid (this was a bigger issue 12 years ago than today)
Assuming a particular elevation tile used (CDB 1.x allows a client to be flexible on mixing and matching LODs from different datasets based on client needs)
It might require assuming that baking in duplicate models from a generic model library (GeoTypical) are more efficiently rendered than if they were instanced in the client. (I don't believe that is true for our system)
It assumes that glTF allows enough attribution to distinguish one model from another when they are all merged together. I'm not the expert on glTF, not sure how attribution is applied to a set of polygons that represents a logical "model"

So, for the CDB repository case, this doesn't seem interesting, as they will want to edit/refine these models. For a CDB edge case, absolutely this would be helpful. Especially when the assumptions about how the client best handles the data are known in advance. For the CDB performance case, I have a hard time with losing the flexibility.

Note: I am not really a fan of the current tiled zip file structure, so don't read this as a defense of how CDB 1.x stores GeoSpecific models. But we do make use of this flexibility in our use of CDB

Kevin Bentley

I have the same concerns about glTF having enough attribution to replace OpenFlight. OpenFlight can store so much more than glTF currently supports. I do support the idea of creating extensions to glTF to make it more M&S ready. What I'm not sure about is how common it is to use some of the flt record types like sounds, heat maps, etc. In other words, how complex would extensions need to be to support most (90%?) of the CDB users? I don't personally know the answer.

Jerome

I fully agree with not losing the flexibility, and what I was considering is a solution that brings the benefits without having to make most of these tradeoffs.

Assuming a particular elevation tile is used -- although this could be a valid assumption that may sometimes be desired for visualization use cases, there is a middle way approach where either vertex attributes for feature IDs or nodes transforms can still support clamping to arbitrary elevation data.
The suggestion to use one batched model for a whole tile is specifically for geospecific models. GeoTypical models would be rendered more efficiently using geometry instancing, which works best with points vector tiles referencing those models (In CDB 1.x, GeoTypical models are also in a shared folder outside of the Tiles structures, rather than in those per-tiles zip files of models).
glTF (or extensions) should be able to assign feature IDs to a list of triangular faces which are part of the mesh to provide this sub-model attribution, providing both efficient batching and proper attributions.

[cnreediii]

Brian Ford

"hardcoded terrain elevation" "replaced by a single batched 3D model": Please do not hard code elevation into cultural models in a 2.5 D representation unless there is a special case reason. That is obviously required for a 3D cultural models integrated terrain skin use case though ala OWT. In the 2.5 D case, it is possible via attribution to force absolute model elevation using AHGT today, but we don't want all models rigidly fused with fixed relative elevation for most LODs, unless possibly they are fused in some real world way like sharing a common wall in a city block. I think that's taking cake baking too far for the 2.5 D use cases. I'm fine with a zip file of individual glTF models, or some efficient way of re-projecting individual models in a composite model to terrain conform as you describe later (BTW: smart/efficient data compression is almost always a win versus more IO, but deflate compression is not mandated in CDB today. The zip is just a contiguous container with optional compression). CDB today supports numerous conformal projection attribute possibilities inside of OpenFlight models (although many application use cases don't require that level of sophistication today to produce a reasonable visualization), so extending that to glTF is somewhat reasonable (but would now make handling that absolutely required). Let's also remember that not every type of sensor simulation always uses a GPU, and that batch handling/breaking is really somewhat application specific even on the GPU today, and sometimes also in a constrained edge cases based on the technology in use. What's good batching in FSI VITAL may be moderately different than a CAE Medallion, web browser, android device, or other visualization application.

Jerome

Well I think what I am proposing for geo-specific models is somewhat half-way between CDB 1.x (points tiles & zipped files with many OpenFlight models) and OWT / 3D Tiles (an opaque mesh, as could be directily derived from LiDAR scan or photogrammetry point cloud), but preserving segmentation and structure (e.g. using nodes). So in effect the glTF serves both the performance use case, as well as the repository/editing use case via the organization of the features, which can achieve the same goal as the points tiles / separate models e.g. through nodes (I think glTF even support instantiation of the same mesh at different nodes for use of semi-typical models inside geospecific content).

I think this really comes down to the data production (both manual and automated), and editing tools. It seems that it would make it easier both to bring in data from point clouds as well as export to constrained 3D visualization systems. Whole tiles and their nodes might also be editable out of the box in modeling tools like Blender, while CDB 1.x tools could be adjusted to move nodes transforms instead of points, and converters between the two could be implemented.

And of course the point tiles and referenced models could also still be an option for what was traditionally 1.x geospecific data layers, but maybe in 2.x this could actually work exactly the same as the geotypical use case, where all models of a 'package' encoded in a GeoPackage are all in the same table of glTF models, since all of that is already in a single GeoPackage anyways, rather than introducing an extra level of 'models for this tile' or a zip file container.

BTW I would really orient the glTF axis to be tangent to the Earth surface at the center of the tile (ENU coordinates inside the glTF).

Ryan Franz

Here is the list of OpenFlight primary nodes that CDB used.

Header
Group
Object
Face
Mesh
Local Vertex Pool
Mesh primitive
Indexed Light Point
Degree of Freedom (DOF)
The various vertex node types
Level of Detail
Switch

All other primary nodes are not used, and I think that none of the newer OpenFlight nodes were explicitly added to CDB as used. There are also Ancillary and Palette records not listed here. Also, I was always fuzzy on whether the extended material palette was explicitly used or not, but we support it.

Jerome

Well that performance case is partly why I suggest a single glTF for all your geospecific models within a given tile, which in addition of being a single IO, is also ready to load onto a GPU VBO.

For models that are referenced more than once (geotypical models chief among them), my understanding is that you would be re-using the models in multiple tiles of that 85 or 1365 tiles GeoPackage, so many of the models might already be hot in the GPU when you load a new tile. For truly geotypical models, you would likely be re-using the models in tiles spanning multiple geopackages (hence why perhaps a stand-alone geotypical models package might be a good idea).

Also perhaps a single SELECT returning all the models you're going to need for your new tile(s) might achieve more efficient IO... You may also want to preload models of a tile pyramid in advance of using the individual tiles.

As for the layout within the GeoPackage, it is a simple tiles table table (identified by level / row / column) with a data blob. This can work for imagery, coverage, models (e.g. single glTF), or vector data (could be using an MVT encoding, could also be used for referencing GeoTypical models). For 'shared' models, I optionally see a table with an ID and a glTF binary blob, and a stand-alone package for all geotypical models could also exists outside of all packages organized spatially by grouped LODs.

I will need to read more about the concept of significant size :)

Re: 1024x1024, I fear this makes it more difficult to reach the streaming / edge cases if the repository is organized this way. Perhaps we could review the IO performance impact of considering 256 x 256 or 512 x 512 ?

[PresagisHermann]

Anthony from Presagis is starting a OpenFlight to glTF converter in order to explore encoding difference. The initial objective is to do a one-to-one conversion, identifying extensions needed in glTF to encode all info. The initial report on support between OpenFlight in CDB and glTF is here (in a branch at the moment) link

As the prototype progresses, we will look at packing of texture and geometry. We will then look at grouping LODs to better support model edition and updates. Throughout, we will look at encoding to ensure we preserve quick access to only the required data (preserve split of data but try to co-locate data better). We will document finding as we go to contribute to the engineering report.

[lilleyse]

I'm still acquainting myself with CDB and OpenFlight but hopefully I can help fill in any gaps from the glTF side.

The glTF requirements for OWT are:

• Per-vertex and per-pixel metadata
  • Support for multiple feature layers
  • Support for external feature tables
  • Support for material ids (note - currently no support for multiple ids per pixel, but we were inspired by CDB's approach)
  • See in-progress extensions: EXT_3dtiles_feature_metadata and EXT_3dtiles_feature_hierarchy (includes support for feature classes)
  • Despite the naming these are general purpose glTF extensions not tied down to 3D Tiles. We plan on changing the naming to reflect that.
  • GGDM attribution would be built on this
• Orthographic texture support
  • Ability to drape ortho imagery onto a glTF efficiently
  • Implemented in one of two ways: additional texture coordinate sets (supported in glTF already) or an extension that provides projection/transform information for generating texture coordinates at runtime. Still deciding which approach is best for OWT and 3D Tiles

Something that may also be of interest is we're designing a vector tile format built on glTF. Baseline glTF provides point, line, and triangle primitives which we would extend with EXT_3dtiles_feature_metadata for feature metadata, (optionally) KHR_draco_mesh_compression or EXT_meshopt_compression for geometry compression, and styling metadata that could, among other things, allow point features to externally reference other glTFs to do instancing (with EXT_gpu_mesh_instancing as another option). We also might borrow the concept of junction IDs from CDB to handle cross-tile discontinuities for line networks.

I reviewed the CDB and OpenFlight doc and it all looks accurate. I think a single CDB extension that adds any missing gaps is the right approach here.

One other extension to look at is AGI_articulations. Other good ones from the Miro board are KHR_lights_punctual, KHR_materials_unlit, MSFT_lod, KHR_texture_transform.

On the topic of packaging - glTF supports external references to textures and buffer data but not to other glTFs, so while textures and attribute buffers can be loaded on demand it might not meet the full use case of storing node hierarchies separately like for LODs, switch nodes, or exterior/interior. 3D Tiles kind of fills this need since an LOD hierarchy can be expressed in tileset.json, but I think an extension would be required for all of the CDB model packaging needs.

[PresagisHermann]

Just posted an updated report on 3D model conversion from OpenFlight to glTF as the converter implementation progresses.

Original findings are:

• polygons need to be converted to mesh (what is the implication if we want to covert back to poly?)
• LOD node tentatively use and existing extension but it does not support significant size attribution (this brings tool interoperability as we create CDB extensions in glTF)
• Need to review axis convention as they are different

The converted OpenFlight model was successfully loaded in Blender via its glTF 2.0 importer.