Load balancing

[voodootje0]

A few months ago I had a suggestion send to tastybento about storing all island data (from blocks to signs and even chests) and it would be amazing if this can really be added.

The suggestion: I don't know or it's even possible but is it possible to save all island blocks/chests/signs/hoppers to MySQL/Redis?`

With this it's possible to create mega skyblock servers with all the same islands, that can grow over 2500 players.

Now it's not possible to hit over 500 people since the server won't be able to handle it. If you can create something it's possible to share all the data between servers, that would just be amazing.

The answer: Huh, interesting. I see what you want - load balancing. Ideally, I'd spin up a VPS when the number of users became too high, say when the server count hit 300+. Then when it was operational, and the player count went over say 450, I'd start directing new players to that server. Their islands would be pulled from a central store that would be duplicated and placed into the new world just before they joined. Then when they logged out, the island would be placed back into store.

Challenges

• [x] You'd need to use Bungee and Spigot to make this work and be able to dynamically add Bungee servers as required
• [ ] Players would not necessarily be adjacent to the players they were next to previously. This might be okay for skyblock, but for acid island it could cause issues
• [ ] It'd be very tough to handle if players logged out on someone else's island. When they logged in again, they may have to just be sent to spawn all the time.
• [ ] Warping to offline player's island would have to be done in some way, i.e., pasting the island just in time.
• [ ] Saving and pasting islands is computationally intensive, but I'd probably use ASyncWorldEdit and just wait for the islands to be pasted. This could cause some delay before players could enter their island after logging on, but that could be hidden by sending them to spawn first and loading their island in expectation they would teleport there.

Now, I'm not saying I can do this! But it's a neat idea!

[Poslovitch]

Maybe an addon could do it? I'm not sure. But that's really a great idea!

[tastybento]

The biggest issue is actually serializing blocks to store elsewhere. The only successful way I know is to use schematics. So FAWE could be used for that.

[Banbeucmas]

@tastybento As we need to store the block, I think you might want to somehow looking at how CoreProtect does it job? I am not sure if it does store the block Meta so you might have the check it out.

[voodootje0]

I think the best will be when the location of all the blocks should also be saved and placed back at the exact same location. This will make support for third-party plugins that use locations (of course they should also would require some kind of external storage)

[drewg02]

Could you not just create the new islands on the new server after the first server had over the specified amount of islands created? Then go down the list of servers until one had an open spot when a new player creates an island? Then just save everything other than block data in the database. You could also have a main hub server with a spawn, pvp, shops, whatever you want, and make all players log in there and it moves them to whichever server the island they teleport to is on, and use plugins that support /tpa /tpaccept /tp /msg etc. over a bungee network.

[siva-u]

Could per-world islands work with something like this?

[Poslovitch]

https://github.com/boy0001/RedEdit

[Poslovitch]

No ETA yet, since it requires so far to work on a few other features first.

[BOT-Neil]

hypixel made and uses this for their skyblock https://www.spigotmc.org/resources/slimeworldmanager.69974/ it's the perfect solution to bentobox imo, but I have no idea how to implement it. 0_o

[Poslovitch]

hypixel made and uses this for their skyblock spigotmc.org/resources/slimeworldmanager.69974 it's the perfect solution to bentobox imo, but I have no idea how to implement it. 0_o

In our current implementation, I don't think it's good. Worlds are not small, they can get extremely wide quickly, if configurated accordingly. Maybe, and I say maybe, it might come in handy when we'll have "one world / one island", but again, this definitely won't be something a lot of people will be able to use on their servers, as it would require a powerful and scalable infrastructure, which is expensive.

[BOT-Neil]

yes, well in the future a one island one world setup would be better. It would require them to install mongodb yes, but why would they need powerful and scalable infrastructure?

https://hypixel.net/threads/dev-blog-5-storing-your-skyblock-island.2190753/

According to the graph from hypixel their average skyblock size is 67.9kB. So it will save disk space too. I think slimeworldformat is the way to go forward, hypixel has already done it. Buttt yeh that would require major changes right?

[Poslovitch]

yes, well in the future a one island one world setup would be better. It would require them to install mongodb yes, but why would they need powerful and scalable infrastructure?

hypixel.net/threads/dev-blog-5-storing-your-skyblock-island.2190753

According to the graph from hypixel their average skyblock size is 67.9kB. So it will save disk space too. I think slimeworldformat is the way to go forward, hypixel has already done it. Buttt yeh that would require major changes right?

Interesting read there. Thanks for providing that in this issue, it's going to help us a lot when we'll start working on that.

I think slimeworldformat is the way to go forward, hypixel has already done it.

It's the way forward, only if we provide a similar "one island / one world" feature. Again, for existing servers that rely on BentoBox, ASkyBlock or any other Skyblock plugins available, this is not what Slime is intended for.

hypixel has already done it.

Hypixel has a huge, professional development team to back their needs. Here at BentoBoxWorld, we're only two folks that want to have fun and solve challenges, and a bunch of kind contributors that help us go forward 🙂.

[tastybento]

I've experimented with "one island one world" and the problem is that loading worlds is not done in any async way by the server. All the worlds are loaded at startup and during that time, the server is not in use so it's fine for them to take all the time they need. If you try to load worlds mid-game (try it), you'll see them lag the server badly. If there's a way to load worlds async or without incurring the lag, then this approach would work.

Note that this approach is also quite peculiar to games that use teleporting to visit others. AcidIsland for example was designed to use boats to visit others and so wouldn't work well.

[tastybento]

I just read the article - very interesting BTW, and they use custom code to load the world into the server. It'd be nice to know how to do that.

[Poslovitch]

I just read the article - very interesting BTW, and they use custom code to load the world into the server. It'd be nice to know how to do that.

They are open source: https://github.com/Grinderwolf/Slime-World-Manager

[Poslovitch]

I've experimented with "one island one world" and the problem is that loading worlds is not done in any async way by the server. All the worlds are loaded at startup and during that time, the server is not in use so it's fine for them to take all the time they need. If you try to load worlds mid-game (try it), you'll see them lag the server badly. If there's a way to load worlds async or without incurring the lag, then this approach would work.

That's the main issue I think, but maybe SlimeWorldManager could mitigate that. Still, it clearly wouldn't allow people to have gigantic islands. They would just appear to be in their "own personal space", that's it.

And, what made the server lag exactly? Loading the world or creating it? I don't remember when you showed that to me... :x

Note that this approach is also quite peculiar to games that use teleporting to visit others.

Warps, Island Homes, etc... could be rewritten to use a 3DVector going from the island center to the desired position. If implemented this way, that wouldn't be an issue.

AcidIsland for example was designed to use boats to visit others and so wouldn't work well.

That's a funny thing from AcidIsland, but, basically, if you're in a "one world one island", you'll just have a giant empty sea.

What could be a real issue however, are slime chunks. I don't think that Spigot provides the API to mess with slime chunk seeds when creating a world.

[BOT-Neil]

Why wouldn't it allow them to have big skyblocks? It will allow them to have even bigger skyblocks because its so efficient compared to the normal region file its like 10%

Yeh, for acid island that is a bummer you can't swim to others.

Also I think easiest method might be 1 server per island then you could have for example 10 or 1000 servers online. But that would require alot of ram, maybe 500mb per server??? It would work like this each skyblock server puts their status on their motd so for pings it would either say "ready" or "someuniqueislandidentifier" then you ping all of them put it into a list and send people to their island if its in the list or send them to an empty server which then loads their island. I can help with doing the pinging info and sorting part if you guys want to follow on with this, but sadly I think I can't be much more help, and I don't think this is the solution you guys are looking for xD

Unless you do shards of like 10 islands per server which is probably harder but maybe better. I really don't know this is all so complicated lol goodluck thinking about this xDDD

[Poslovitch]

I really don't know this is all so complicated lol goodluck thinking about this xDDD

That's mostly why this issue is still open for 2 years.

Why wouldn't it allow them to have big skyblocks? It will allow them to have even bigger skyblocks because its so efficient compared to the normal region file its like 10%

It's written on SWM's Spigot page that "it is not designed to handle big worlds". So I guess they're talking more than 2k*2k there.

[ShadowChi1d]

Would love to see the progress of this! I implemented it into a basic "area" style PVP game, but could never get it to work 100% the way I wanted it to.

[Guillaume-Lebegue]

I was looking into the way to save the island and found some things.

I think we could use something like AsyncWorldEdit. Since it implements Schematic loading and saving. Here is example of it https://github.com/desht/dhutils/blob/master/Lib/src/main/java/me/desht/dhutils/TerrainManager.java

So we could save an island when all members are offline. And when someone login when their island aren't loaded they would be tp to the spawn. And we would start the loading of the island and display a nice message when done.

[Poslovitch]

I'd agree with the use of (F)AWE and I'm already planning to integrate it into BentoBox. However, we've always tried to follow a "no hard dependency" rule. Making (F)AWE mandatory just to allow load balancing sounds a bit overkill, although I have to admit that load balancing, if it ever gets implemented, would be a huge thing.

We already have a way to store islands, and it works fine : blueprints. They're nothing more than a JSON file that maps to each block's "position", the block's serialized BlockData. Entities require a bit more work. Yet, the fact that Blueprints are just some sort of "hidden" JSON files make them already entirely compatible with our database framework. There's technically no additional thing to do:

• When the player logs off, you save the island into a blueprint file and then delete the island from the world
• You save that blueprint file on the database as a "normal" database entry
• You retrieve the database entry that contains the blueprint file
• You paste the blueprint file into the world

It's obviously way more complicated to implement load balancing than what those 4 bullet points might make it sound. However, I believe that through the use of our already existing "Island serializer" (which is what a Blueprint is, in the end) we could avoid some painful code rewrites.

[Guillaume-Lebegue]

I was looking at how it was done to delete chunk in bentobox. That doesn't seem to be async but just spaced in time. Is there a reason to why? Because that could pose some problem when pasting and cutting island like we need here.

[Fredthedoggy]

A way to make it much simpler then trying to save islands to a database, would be to have a server (Like BSkyBlock_Storage) That stores all of the islands, the same way a regular bentobox world does. The difference, is when someone logs in it would copy their world out of storage, and into the new server, and leave it there. When the person logs of the world would be copied to the BSkyBlock_Storage World, their island's chunks would be cleaned.

[tastybento]

There is no async way to delete chunks now. Bukkit used to have a regenerate chunk method that was really good - it was async and you could literally tell it a million chunks to regenerate within a tick and it would just do them. It had some problems in that the client would see some artifacts if they were in or near those chunks, so I had it so that deleted islands were not used until the server was rebooted. In 1.12 it became less async and starting to be somewhat problematic. In 1.13, it was removed by md5 because the regeneration couldn't regenerate exactly because there are cross-chunk things like trees etc. So, now the only way to delete is to use the Bukkit API and write air blocks.

Now, recently, I looked at writing blocks fast again using NMS. I even wrote some code for the Blueprint paster to try and do this because in theory, writing blocks fast is really what's needed. I actually couldn't get it to work so whatever recipe I was following wasn't correct, but if we want to get back into NMS again, then this might be the way to do it.

[MrTalon63]

So it's possible but problematic?

[kit8379]

Push

[portlek]

i recommend you to use slime world manager via a skyblock plugin(idk if bentobox supports or not). then store the whole file in MongoDB GridFS.

[IllusionTheDev]

I wrote an entire Skyblock Core around this concept -

If absolutely needed, you could just save it in minecraft's own format (.mca) - Although the files are somewhat large, don't work cross-version, and have tons of other issues, it works for basic Skyblock servers. The real issue is AcidIsland, but with some funky math (and hidden teleports, maybe?), we could use the 512x512 .mca file size to our advantage.

[Treppenhaus]

Any updates on this? I would love to see something else instead of "Skyblock Server 2", "Skyblock Server 3" with about 150-200 Players each.

[ghost]

It's written on SWM's Spigot page that "it is not designed to handle big worlds". So I guess they're talking more than 2k*2k there.

We're currently looking for a plugin to make a new OneBlock server in our network. While I do not have too much context in this issue, one idea I had in the past was creating a world for each island (we were thinking of developing a multi-server OneBlock game mode, something similar to what Hypixel does with their SkyBlock where lobbies, certain map zones & private islands are completely separated from each other).

Our current OneBlock server with BentoBox takes up almost 415 GB (which is really inefficient if you ask me). That's ¼ of the total storage of our dedicated server (2 TB in total). Purging islands does not help in any way (and it even crashes the server when trying to do it).

It's worth noting that this is a 1.16.5 server and BentoBox no longer supports it (and thus it is also one of the reasons we want to make a new one). I'm also unsure whether purging islands helps with storage usage in recent BentoBox versions.

root@nodo2:/var/lib/pterodactyl/volumes/3d24a5e0-5811-42f4-9bbd-b3b8cd2a7f5b# du -sh ./*
4.0K    ./airplane.air
267M    ./ArenaPVP
20K     ./banned-ips.json
36K     ./banned-players.json
4.0K    ./bukkit.yml
97M     ./cache
4.0K    ./commands.yml
257M    ./crash-reports
15G     ./EndMod
4.0K    ./eula.txt
23M     ./evento
4.0K    ./help.yml
3.2M    ./libs
15M     ./libyjpagent.so
4.0K    ./log4j2.xml
85M     ./logs
265G    ./oneblock_world
122G    ./oneblock_world_nether
4.0K    ./ops.json
12K     ./paper.yml
0       ./permissions.yml
18G     ./plugins
54M     ./purpur.jar
28K     ./purpur.yml
4.0K    ./server.properties
1.9G    ./SpawnOneBlock
1.5G    ./SpawnOneBlock_nether
2.2M    ./SpawnOneBlock_the_end
46M     ./spigot.jar
8.0K    ./spigot.yml
4.0K    ./tuinity.yml
108K    ./usercache.json
4.0K    ./version_history.json
4.0K    ./wepif.yml
4.0K    ./whitelist.json

[tastybento]

If you are starting afresh then all you need to do is run the purge command regularly and restart the server. This is best done daily so that new users will take the emptied spots. Purge does not actually delete chunks from the world file, it just makes space for in-fill.

From what you write, I see two issues:

1. Your current large world size on disk
2. Trying to keep the new server you plan from becoming too big.

If you don't plan to keep the current world and start a new server then then 1 is not an issue.

If you regularly run purge and restart, then 2 will be solved.

The one-world-per-island thing sounds cool but actually it would end up taking more space in all likelihood because a world file is bigger than just the chunks within a world file. That said I do know they can be stored compressed but the loading/unloading and handling things like warping between islands is tricky. The single world is actually quite efficient in that only the chunks needed are ever loaded. The issue you had was that the world disk size grew very big due to a lot of islands being made. Anyway, thanks for the note.