Improved, flexible map format

[aschaeffer]

Map Format (JSON + Binary Data)

• Almost everything (textures, mapvars, entities, ...) is stored in a JSON datastructure except the octree and lightmaps
• Octree and lightmaps are stored in extra files in a binary format
• JSON, octree and lightmaps tar'ed together and compressed (lzma, xz)

[Croydon]

tar? Zip has a much better support and it's already supported by Inexor/Sauerbraten. I would rather use zip.

[a-teammate]

lzma is also supported by Inexor see https://github.com/inexor-game/code/commits/newmapformat but i agree, i would prefer zip as well. bc its performance is better and filesize isnt such a big matter for maps.

For content-packs on the other hand lzma could be useful, since they could get bigger. We probably have to test both configurations. (which is not a big deal, e.g. just replace opengzfile with openxzfile)

[aschaeffer]

tar is a unix command which concatenates multiple files into one file without compression. This single file can be compressed afterwards. Therefore you can tar some files, then use your favorite compression tool. Zip just does both steps at once and the compression algorithm is fixed.

But that's not the point of my post: The point is that we internally have multiple files for multiple purposes (a file for each: the octree, the mapvars, the entities, the textures, the lightmaps, ...). So you can import and export only specific things. For example you can export the entities and import them in another map. If you want to provide a "complete" map, you just include everything.

The compression thing is on top of the logical structure of the map format. In fact the compression algorithm is only a side note of this ticket.

[Calinou]

Currently, Sauerbraten, Red Eclipse and Tesseract uses gzip, not DEFLATE (ZIP's compression algorithm).

[Croydon]

Small note, since I have explained this now multiple times: Currently Inexor/Sauer is saving map vars in the .ogz file. This also happens if you have created additonal a .cfg file. So e.g. the skybox attribute is now in the .ogz file and in the .cfg file. The .cfg file will overwrite the values of the .ogz file, still, for some reason Inexor/Sauer is already trying to load the skybox, when the .cfg file isn't applied yet. Result: You need to update attributes like the skybox in the .cfg file AND in the .ogz file, if you don't want to have warnings (when the skybox can't get loaded). If you first update the .cfg file, you can type ingame: /skybox $skybox; savemap No joke, actually.

This odd behaviour will be automatically fixed with the new map format, thanks to it's seperating of variables and binary stuff.