search

CesiumGS / 3d-tiles

Specification for streaming massive heterogeneous 3D geospatial datasets :earth_americas:
2.14k stars 469 forks source link

Package tileset in a single file #89

Open pjcozzi opened 8 years ago

pjcozzi commented 8 years ago

Like MBTiles as discussed with @mramato offline.

CC #9.

mramato commented 8 years ago

My first cut at this is fairly straightforward and naive.

  1. A single table, called media with two columns, key, which is the relative path from the root of the tileset, and content which is the payload as a blob.
  2. Compressible payloads, like b3dm and JSON, are always stored in the database pre-gzipped. Textures or other pre-compressed data are stored as-is.
  3. tileset.json is stored in the same table along with the rest of the data.

You could get pretty far with the above format, it's main weakness is that it's overly generic and hard to do any sort of validation on without grabbing tileset.json and trying to walk the entire tree. It's also the least efficient for tile retrieval because key is a string and not what you want to be using as a primary key.

A better solution would be to have a database schema for each tiling scheme, which would rely on a well-known types as discussed in #92. For example, a quadtree tree would have x/y/z columns that form a composite primary key; which would make DB look-ups a lot faster. You would still need a generic table for hosting external textures or other shared data but the tiles themselves would be much more structured.

Also, rather than store tileset.json as a pre-generated file in the DB, we could also have a metadata table which stores the tileset in a deconstructed schema. It would then build tileset.json on the fly from that data. This would allow for tileset.json subsets to be generated from larger datasets and possibly other types of dynamic data generation (like merging to tilesets on the fly?). Since we need a generic media table anyway, we could probably get away with not doing this until a need arises.

One last idea would be to store the batchtable data in a separate table on it's own (either denormalized and duplicated from the payload, or by generating the final tile with batchtable on the the fly) In both cases you can then use this for server-side or non-visual queries (either for analysis or dynamic tileset generation). The latter would also allow for clients to choose which data they care about and decrease payload size by not sending down metadata or payloads they don't care about.

pjcozzi commented 7 years ago

Still not official spec yet, but there is a tool to generate this in 3d-tiles-tools. Use tilesetToDatabase.

mramato commented 7 years ago

I submitted #184 which depending on the outcome may impact the format of .3dtiles. If we don't want a standard entry point for all tilesets, we'll at least need one for the db format.

pjcozzi commented 6 years ago

@mramato is it realistic to scope this for perhaps 1 day to include in the 1.0 spec? Or would we be committing to this in haste? Perfectly happy to keep this separate from 1.0 for now.

mramato commented 6 years ago

Since this is just a way to package a tileset into a single file and would not have any impact on the spec itself, I'm fine with making this post 1.0 just to decrease scope.

mramato commented 6 years ago

Hit comment too soon.

However, it would also be trivial to implement it for the 1.0 spec and then simply updated it when 3D Tiles spec updates. For example the DB schema will definitely change for known tiling schemes (which I assume aren't 1.0?) It all depends on how important we think a "single file" format for 3D Tiles is to have.

pjcozzi commented 6 years ago

Pushing post 1.0 for scope.

pjcozzi commented 5 years ago

There is also a GeoPackage Community Extension for putting 3D Tiles into GeoPackage.

pjcozzi commented 4 years ago

Now duplicate with #422.

Before closing this, would want to document https://github.com/CesiumGS/3d-tiles/issues/89#issuecomment-227446949 as the old unofficial .3dtiles format.