Use sqlite instead of json for persistent storage.

[oddlama]

sqlite would be a better fit than json for persistent storage (reliability, speed). Would need a proper framework though to avoid a lot of aweful boilerplate code.

[oddlama]

Even better: Store world-bound data in the world. For example portals should be stored similar to how villages are stored. Completely contained in the world and discovered on startup.

[ryantheleach]

I love you are wanting to store stuff in the world file!!! So many plugins store world data in the plugins folder, it makes it a pain when regenerating the world after testing.

[oddlama]

This is one of the things that should have been possible via the API from the beginning but sadly weren't. Even now it still is challenging to do correctly.

I'm still unsure how to handle global data properly. Things like for example a global list of friends could be manageable, but it gets complicated when we are talking about a /spawn location which could be in any world and may or may not be loaded at a later point in time. It's probably a reasonable compromise to leave them in a global storage for now, but I really don't have a good solution for the spawn location at the moment.

It's also definitely harder because worlds can be loaded or unloaded at any point, so any internal caches or acceleration structures need to be kept in sync. Should be manageable though by keeping a separate set of acceleration structures for each world.

What's more challenging is making it fast. I wouldn't like to store a list of 200 portals each time a single portal is modified - just because a global list is modified and the API needs the whole list to update it. Even worse, portals need to store a list of associated blocks for each portal. This can get out of hand quickly and should be thought of beforehand. Maybe store a list of uuids, and associate portal data separately to each uuid.

Additionally, I'm not sure how Spigot's PersistentData API handles storing things in the metadata of the world internally. If the metadata is always stored as a json string, all the efforts from above would be for nothing. Hopefully the serialization is only applied on saving, then everything is fine.

I must definitely test this.

[ryantheleach]

Under sponge we didn't screw the dimension layout under the guise of multi world compatibility, so you'd just store it in the primary world folder.

This gets awkward under spigot, but I think it's still the best compromise we have.

[oddlama]

actually a good idea. so probably {main_world_folder}/vane/storage.json. I suppose the main world is defined by the level-name setting in server.properties?

[ryantheleach]

I'm not sure if it can be override by the Spigot API, but yes, that's what I would expect as an admin, unless multiverse or something had a configuration setting and had the ability to override it somehow.

Make sure not to forget about the 'world container' command line argument that spigot can be started with. https://hub.spigotmc.org/jira/browse/SPIGOT-5824?workflowName=Jira+Fixed&stepId=1

[karthanistyr]

json storage in world folder is a very good idea. I'm striving for a read-only plugins folder for my deployment routines.

[ryantheleach]

After seeing the bump, I re-read the thread.

We should likely have 2 configs in the primary world, 1 for it's dimension if applicable, and 1 for the stuff shared for it's universe.

(read as, global storage related to worlds, unless something seriously changes regarding API around supporting multiverses.)

This separation will also make it clear to admins which data will be lost/abandoned/need migrating, should they delegate the main world to a different world.

All effort should be made to store data per dimension though, unless it doesn't make sense, or would otherwise provide integrity issues.

[oddlama]

Since vane is now using each world's PDC to store related data, I consider this done :) Global data is still saved via json like before, and as I currently cannot come up with a significantly better solution I guess it will stay until I can think of something. Guess that would be a new issue though.

[karthanistyr]

Thanks, Can you clarify what’s the status now on saved data, I don’t think I understand the jargon yet (eg « PDC »)

[oddlama]

Sure! Beginning with the next release, any data related to a specific world (like a portal or a region) will be stored in the level.dat file of that world. This is called the world's persistent data container or PDC for short. This functionality is provided by the bukkit server software, and therefore it's guaranteed to be supported for future versions.

Anything that is not related to a specific world (region groups, permission settings) will still be stored as usual, as it is global data that cannot be assigned to a specific world. There is no equivalent PDC for global storage, so we have to use the json storage system for that. Although there isn't much global state in vane except for the things mentioned and i guess /setspawn.

EDIT: One more thing: Conversion will be happening automatically. If you need to edit stored data at any point after the conversion, you can still do so by using an NBT editor like NBTExplorer.