Procedural Generation Performance (Question on issue #80)

[JaroDevelop]

On Issue #80 you stated that all Currently rendered asteroids are loaded in all the time. I decided to do a question issue instead of replying on that thread. If all currently rendered Asteroids are loaded for all players all the time Then would reducing the player count Help with performance? I only have 1 instance of world 1 server which all players will be playing on at once. I want to know if I should lower the player count now. I want it to be at least 28 or 30 players total if I have to lower it. Currently its at 42 Which I can try to go down on. Is there anything else I can do to help performance of the procedural generation's load? I already have voxel reset on and apparently concealment doesn't work for this? What would be the right amount of rings/ belts per player count so I know what I can work with. And Yes I know that sizing matters when it comes to how many asteroids are loaded per player's view bubble. I want at least 2 rings and 1 belt though if you think I should go smaller, I just want your thoughts on what I should start out with. Then I could possibly do a playtest to see if the server can handle it. EDIT nvm UD is not the biggest one my bad. So it might turn out to be the same for mine - Jedd

[JaroDevelop]

I was thinking if you could see code or how the asteroids are concealed you could do a fork of the plugin or gain knowledge from the code of ways you can get an idea of how concealing works with vanilla generation of asteroids and possibly find a way to improve performance for SEWorldGenPlugin? Its at least worth a look. Hey it looks Like It wasn't Concealment plugin but a Torch Command Labeled myvoxelregen that's responsible for concealing asteroids. When I run the command it filled in the Asteroids voxels that I dug up which were generated by the SEWorldGenPlugin. Apparently its also supposed to conceal the asteroids after the voxels reset too. But that wasn't happening on UD which I am sure has Voxel reset as well. To be fair it probably wasn't concealing on ours either. So you might want to have a look at how that command works. I'm trying to find that in the Torch git and I can also DM you our automation to see what it does during the reset itself.

[thorwin99]

The problem with asteroids, as stated in #80 is, that normally they generate client side and are synced by deterministic seeds of each cell. That doesnt work here. So every asteroid generated ( that is visible to a player) needs to be synced and the only way currently is to force them to do that by setting the asteroid to sync to ALL players. I cant selectively do that. So the server generates all asteroid that need to be generated, either for players or grids.

To answer your question, i dont know. I have never tested or used it for such large player counts. There is currently no way to really improve the performance of generating asteroids with the plugin, sadly. I would use a low density and low view distance, which causes a minimal amount of asteroids being loaded at once but i cant be really specific about it.

The torch stuff works, because Torch has a client side that it communicates with. The plugin doesnt

[JaroDevelop]

So if the torch stuff normally works with the client as you said. Why does Voxel Reset command in Torch actually work with the asteroids generated by your plugin? The command Partially works with your plugin is why I think you should try looking at the code for it. Right now it just resets the voxels for your asteroids along with vanilla planet voxels. Again I think it might be worth a look.

EDIT: -Continuation and Explanation

Apparently It Works Fully and does not do client side generation as a dependency. Please try to take a look at the code, and I'll DM you the automation so you can see what it does server side if you need it as a reference.

What it does is so it basically deletes all the Asteroids that are loaded and the player has to "regen" them by moving near them and its also automated.

I also noticed also it borks Persistent GPS' making them blink every second I have to stop the server and turn all Gps' to NONE to get it to stop. It also does the same thing with dynamic GPS' until you move next to a different part of the belt.

So it does work I think it does something completely different from concealment because it effects the gps' because it deletes the asteroids that your plugin spawns. It saves its gps location and blueprint of the Asteroid, (model w/e) before hand probably when it was loaded the first time. Then Re-paste's the Asteroid when a player is in loading distance.

It has nothing to do with generation of the Asteroids themselves it just saves, deletes, then re-paste when a player is nearby

[thorwin99]

Yes, it does work since it runs both on client and server. Why does it work with the plugin? The plugin just spawns the asteroids into the world, as entities, that are forced to get synced to clients. The Voxel reset tool now simply resets those asteroids on the server side or rather, simply deletes them. As far as i understand, the plugin then sends a message to the client mod to do the same? Regardless, the plugin already closes each and every asteroid not in view distance except for edited ones. So asteroids not rendered, where no player or grid is near them, gets closed. The reason why grids are included is simple. Travelling in a ship does remove your player entity from the world? for some reason. Asteroids would not generate so grids also need generate asteroids.

I dont know why it affects gps generation, since those are generated independently from the asteroids and stored in some class torch doesnt even know off.

The reset doesnt save asteroids, it just straight up deletes them and allows them to regenerate. Its just usefull for edited asteroids, to reset them and make them editable again.

[JaroDevelop]

So grids on asteroids acts as a constant loading zone for other asteroids? So they have their own view bubble set aside from players. It works the same way though? Is there a way to affect only Grid View distance? Or are they saved to the same setting for both players and grids? If the two are separate Maybe we could have grid view distance Lower than the players? Or just have view distance off for static grids? That is if there's a difference between static and mobile. EDIT: yes I know Theres just 1 setting in the dedicated/Torch World Settings I'm just wondering if it that setting can be modified and if it calls for both player and grids differently in the code when it sets the view distance. I recall that sometimes the grid view distance gets glitched and no asteroids appear and you have to get out of your seat to load asteroids in. So maybe it does?

[thorwin99]

Grids have something called a player presence tier. If that goes to low, eg no player is in distane AND hasnt been there, the grid unloads the asteroids. So only when players are near, asteroids are generated. Sorry that i forgot to mention that.

[JaroDevelop]

I guess currently the best way to save performance then in asteroid fields is to prevent people from afking for too long at their bases or ships. But if everyone has a base in a field and are playing at the same time as each other that can impact performance a lot. Also for UD -Upsidedown said that only a few players were loading in roids. But it was loading the asteroids for every player that was online as well, that's the cause of the performance load? For Example if only 3 people were in the field and each had 10 asteroids near them then everyone on the server had 30 asteroids loaded in at the same time for everyone on the server? So if That's the case you want as little asteroids loaded per view bubble as possible. However finding just 1 asteroid on the screen will be hard to pick out. Another idea came to mind If you generate every asteroid in the belt all at once so they are technically already there. Then you turn off SEWorldgenPlugin would it revert back to Vanilla processing as in it only loads the asteroids client side for the player?

[thorwin99]

First, the spawning of asteroids currently needs to be done that way, that every player loads them. There is currently no way to do that without it, at least as far as i know. Generating every asteroid once is simply not possible. That was the first approach i took when developing the plugin and it would have taken days to completely generate the belt beforehand. Also those asteroids would also always be loaded, since the saved flag for them indicates that they are generated and should be shared with all players.

[JaroDevelop]

Another Idea came to mind I'm not sure if you tried this or not yet. But if you could allow Vanilla Generation only for certain parts of the map would that work? That way it would work with the vanilla loading. Although I think that would be pretty hard to implement because the only option for it right now is infinite and changing it to load in a specific area might be a project in its own right. Edit: that would also have to be worked with client side right? Correct me if I'm wrong but syncing locations and use the vanilla generator between Server and client might be too difficult?

[thorwin99]

Nope, again, the clients generate their own asteroids on vanilla mode. it is possible to patch the vanilla generator with harmony on the server side to do that, but the client side would still see them everywhere

[JaroDevelop]

Would you still be able to see any asteroids if client side was turned off and the vanilla generator was patched like you said? Because if the Client side asteroid's density is 0 and the server side was patched to be a different density at a certain location would you still see asteroids everywhere? And have it not have the actual roids on the server show up on the players screen? oh wait it would still need to be synced somehow damn.

EDIT: Getting it to sync with it on server side would either need a client mod or a plugin to be installed to communicate when the asteroids should appear on the client. I guess you could do that with 0harmony installed the way you install your plugin client side? also you would need to make it easy to install for everyone which is a big task. I was just thinking just by joining the server would actually be too hard too do.

[thorwin99]

No you would just end up with the same situation the plugin works, so you would end up with the plugin generator, just as a patches Vanilla generator, the rest would be the same.

Exactly. Syncing it with client and server is an issue without a client mod or rather plugin, since mods dont have access to asteroid spawning. However, i just dont think that making a plugin a requirement to join a server is a good idea, since plugins are way way more powerfull than mods, and even though it would not be harmful, i would not want to force users to do it, If a server wants to use the plugin then it should be without a client plugin, thats just my thoughts on that

[JaroDevelop]

The only other way I would think to bypass Would technically have the asteroids side-loaded on a different server. Being the asteroid field IS a different server and the other locations of the map would be on different servers. Do you think there a way to trick a certain location to think its a separate server? Which is what I Think Sigma Draconis does but I think they use a mod or something for asteroids, not sure. EDIT: I think the best way to describe this way of thinking is old Video Game loading zones? Like I am not sure how hard it would be to make a "Loading zone" while it either transfers you to a different server or tricks the dedicated client you are being transferred in this case.

[thorwin99]

I dont really know. Never was an expert on SE server stuff.

The plugin works for Multiplayer, but i really dont think asteroid generation works well in MP when there are so many players, without lowering the density significantly or reducing the view distance, sadly

[thorwin99]

Issue will now be handled on the MultiplayerSupport branch. There might be a solution to the asteroid generation performance impact in multiplayer by creating a mod, that generates the asteroids for the plugin on the client. That mod would communicate with the plugin, to generate the asteroids. This would remove the streaming issues, since streaming thousands of asteroids is heavy on performance of client and server. However, i dont know if that will work yet, since the Mod API is really restrictive.

[dlockman]

Seems like it would be worth seeing how Patrick is generating asteroid belts in StarSystemgenerator? That seems to both be decently performant and not have Jumpdrive-into-asteroids issues. Unfortunately that mod only supports random generation, whereas this plugin allows far more customization.

[thorwin99]

He utilizes the mod-API, which is severely limited in its asteroid generation capabilities. Firstly, the asteroid generated by it cant be temporary, meaning every asteroid generated is saved to disk. Thats an issue, as the plugin allows very dense and large asteroid fields, and having tens of thousands of asteroids, that might not even get mined ever, to be saved to disk just wont work. This is actually the most important reason i decided against using it, as I need temporary asteroids for the plugin, that are deleted if they havent been mined / build on and no player is near it. This saves ram, and storage.

Secondly, his implementation automatically syncs to clients, since saved asteroids are always synced, just like the plugin. So that wont change with his implementation, and wont improve performance.

Technically, the important difference between his implementation and this one, is the amount of asteroids generated and saved. His saves any asteroid AFAIK, since the mod api is limited to that. This means, asteroid fields need to be less dense and frequent. You can effectively achieve nearly the same performance, simply by limiting the asteroid density to 0.1, i think, as that is the lowest the plugin allows. Still, there is some overhead in the plugin to allow the temporary asteroids to exist and be synced.

I was actually quite interested when his mod came around, to see how he would tackle the asteroid generation part, and if i can improve this one. But in the end its a compromise and while i could rework the plugin to work like his mod, i would simultaneously limit the capabilities of the plugin.

I would love to improve asteroid generation performance, but really, the only way i see to improve it, would be to have the client and server uncoupled in its asteroid generation. This means, the client generates all asteroid it needs, and the server generates all of them, but they wont need to be synced by sending the information to the clients. This is how it is done in vanilla, but is only possible, if the server can guarantee that every player has the plugin installed.

[dlockman]

Ah, I wasn't aware you had already looked into that. Thanks for the info Thorwin!