webbukkit / dynmap

A set of Minecraft mods that provide a real time web-based map system for various Minecraft server implementations.
https://www.reddit.com/r/Dynmap/
Apache License 2.0
2.03k stars 414 forks source link

LevelDB storage type #4066

Open ashduino101 opened 5 months ago

ashduino101 commented 5 months ago

Feature Description: An option to save map tiles and data in the LevelDB format would be nice. This would allow for the data to be stored as a collection of moderately-sized files in a single flat directory instead of millions of individual files (filesystem), a database that probably supports a lot more functionality than is necessary, degrading query performance (SQLite), or a database that is stored at a low filesystem level and is difficult to migrate or control, as well as the necessity for a socket to be running (MySQL/MariaDB/PostgreSQL). For the database keys, perhaps a format similar to Bedrock Edition's world format would suffice, where tile coordinates are serialized as int32s along with an indicator to differentiate the tiles from other types of data being stored (e.g. entities, dimensions, general metadata), a dimension ID, and maybe a map type ID as well, just as an example.

Additional context: In case you don't know what LevelDB is, it's essentially a glorified on-disk dictionary, with extremely fast query times for large amounts of binary rows. MCBE/MCPE uses it, allowing for extremely small world sizes (some of the fairly large example maps are under 8 megabytes!), low memory usage, and fairly succinct filesystem storage.

rautamiekka commented 5 months ago

I'd just go with H2 instead if SQLite won't do, since it seems to have less problems with performance than SQLite.

^ According to https://stackoverflow.com/a/62845028 it has a high-likelihood of corruption if the app shuts down illegally, though.

VL4DST3R commented 3 weeks ago

This would be awesome as I believe it will indirectly address my issue with having a single huge monolithic db that is a pain to backup and maintain.