KoBeWi / The-Soulhunter

A MMORPG game prototype made in Godot game engine
MIT License
112 stars 10 forks source link

Can this work for Godot 4+ games with multiplayer and open world chunks? #2

Open WithinAmnesia opened 4 months ago

WithinAmnesia commented 4 months ago

Test chunks link: https://github.com/WithinAmnesia/ARPG/discussions/15

I'm trying to find a way to seamless load and unload chunks for a 2D multiplayer game project to make an open world with a working server using Godot 4.2.1.NET.

How can this work for multiplayer and what is needed for this to potentially work? What options can be used for chunk loading and unloading seamlessly in Godot 4.2.1.NET? Please give feedback.

KoBeWi commented 4 months ago

This project was made in Godot 3, so it would need to be upgraded first. It doesn't use Godot's multiplayer system, but a custom one. There is a support for loading/unloading instances (rooms), but it's rather simple and easy to replicate.

IIRC the PDF explains the inner workings: https://github.com/KoBeWi/The-Soulhunter/blob/master/Paper/The%20Paper%20Eng.pdf

WithinAmnesia commented 4 months ago

Here is an update since I last posted. https://jonathaneeckhout.itch.io/jdungeon For the goal right now it is easy to play this but imagine it has seamless chunk loading and unloading. As instead of the black world boundary a new chunk loads in seamlessly. What are your suggestions and thoughts?

Update: https://github.com/WithinAmnesia/ARPG/tree/ARPG-Infinite-Worlds Here is the 128x128 with 32x32 pixel tile chunk to test.

My initial testing seems to feel even faster combat with the 128x128 chunk. This 128x128 chunk has the same amount of entities as the 256x256 chunk. For I moved all of the entities over into the 128x128 chunk. This 128x128 chunk is a good test for using the Infinite-Worlds using the BinarySerializer for Godot 4.2+ from Theraot: https://gist.github.com/theraot/31515e28e2d8bfea33f6c6d5bcd852f6 . There needs to be some testing how to make the chunks seamlessly load and unload. https://gamedev.stackexchange.com/questions/209002/looking-for-help-godot-4-multiplayer-seamless-open-world-chunks

KoBeWi commented 4 months ago

I can't run the project, nor open it in the editor. Getting some error spam about tiles.

If you want seamless loading, if you make your chunk be size of the screen, you should be loading 3x3 grid of chunks, because when going between chunks, you will see multiple of them at once. As for unloading, in my project I had a timeout (I think 1 minute?). If no player was in the room for that time, it was unloaded. Though if the player is moving fast between chunks, the timeout could adjust dynamically to unload far chunks.

WithinAmnesia commented 4 months ago

This project was made in Godot 3, so it would need to be upgraded first. It doesn't use Godot's multiplayer system, but a custom one. There is a support for loading/unloading instances (rooms), but it's rather simple and easy to replicate.

IIRC the PDF explains the inner workings: https://github.com/KoBeWi/The-Soulhunter/blob/master/Paper/The%20Paper%20Eng.pdf

Oh excellent. What are your thoughts and feeling and ideas for what should be done for seamless chunk loading and unloading now with this update and promising code from Theraot? I hope you can read this code better than I can Oh 1 sec.

WithinAmnesia commented 4 months ago

I can't run the project, nor open it in the editor. Getting some error spam about tiles.

If you want seamless loading, if you make your chunk be size of the screen, you should be loading 3x3 grid of chunks, because when going between chunks, you will see multiple of them at once. As for unloading, in my project I had a timeout (I think 1 minute?). If no player was in the room for that time, it was unloaded. Though if the player is moving fast between chunks, the timeout could adjust dynamically to unload far chunks.

error spam? What does this look like? I had to re-import stuff lots and there has been unable to write / lack permissions before but it has also been an issue of too long directory. So try a short directory path too, it might help? What system and stuff are you using I might be able to help get it running? All feedback is good.

KoBeWi commented 4 months ago

https://github.com/KoBeWi/The-Soulhunter/assets/2223172/d9bbec1a-3f74-4068-aa40-af276d14dd7c

This, forever. (the tile errors begin a bit later)

WithinAmnesia commented 4 months ago

powershell_zF6ApB7oHo.mp4

This, forever. (the tile errors begin a bit later)

Interesting. oh! https://github.com/jonathaneeckhout/jdungeon/tree/d941345fd187e51d7d51d24027838baae3ba4b78 this is the JDungeon main branch with the in sync version that the ARPG demo is based from its like 99% the same minus its 256x256 and the test is 128x128.

WithinAmnesia commented 4 months ago

Did I corrupt the ARPG 128x128 version? Hmmm. I uploaded it and downloaded it from its own github branch and tested it and it worked? But its supposed to work on not just my pc so I wonder what the issue is. Can you run the regular JDungeon from that link? If so it must be an issue with my 128x128 version. Maybe the .godot/imported section hmm.

KoBeWi commented 4 months ago

What Godot version exactly?

WithinAmnesia commented 4 months ago

What Godot version exactly?

Godot 4.2.1.NET https://godotengine.org/download/windows/ or linux / mac .etc

KoBeWi commented 4 months ago

Ok I managed to run it, but only the server (the client couldn't log in for some reason).

Looks like your chunks are very big (I mean compared to the screen size). Not sure how a 3x3 grid would behave, maybe a better option is loading chunks when a player gets close to the border. There is space for that.

Also for chunked maps I'd suggest creating some specialized editor, because editing them separately is going to be a hassle if they are supposed to link seamlessly.

So what do you expect from me exactly? I already described the state of this project. I can give some general advice, but I'm not a networking expert. I didn't make any multiplayer game since I abandoned the current Soulhunter.

WithinAmnesia commented 4 months ago

I updated the main page with start instructions now. https://github.com/WithinAmnesia/ARPG/discussions/15 How to start: Run 3-4 instances in the debug -> run multiple instances. Then run the first instance as Gateway. The second instance should run as Server. The third instance as Client. Make an account then log in. The fourth and more instances can be for client multi-boxing testing and work the same as the third instance.

Ok I managed to run it, but only the server (the client couldn't log in for some reason).

Looks like your chunks are very big (I mean compared to the screen size). Not sure how a 3x3 grid would behave, maybe a better option is loading chunks when a player gets close to the border. There is space for that.

Also for chunked maps I'd suggest creating some specialized editor, because editing them separately is going to be a hassle if they are supposed to link seamlessly.

So what do you expect from me exactly? I already described the state of this project. I can give some general advice, but I'm not a networking expert. I didn't make any multiplayer game since I abandoned the current Soulhunter.

WithinAmnesia commented 4 months ago

Ok I managed to run it, but only the server (the client couldn't log in for some reason).

Looks like your chunks are very big (I mean compared to the screen size). Not sure how a 3x3 grid would behave, maybe a better option is loading chunks when a player gets close to the border. There is space for that.

Also for chunked maps I'd suggest creating some specialized editor, because editing them separately is going to be a hassle if they are supposed to link seamlessly.

So what do you expect from me exactly? I already described the state of this project. I can give some general advice, but I'm not a networking expert. I didn't make any multiplayer game since I abandoned the current Soulhunter.

Oh well I just need some advice and like thoughts and what should be pursued? this is a complex issue and I will make a lot of errors so like if any help and or guidance can be give n that would be great.

I just need need like testing and what to do next for I'm not really goo at this but if eel with the right people we can find the solutions together. Once we find what works then everyone can benefit too. So it is important for myself be open minded and like I just need some ideas and knowledge and like any suggestions help too. Its really hard to find propel who are good at this stuff so any advice would help. Any thoughts and questions and insight and like what would make sense to do next with this big puzzle would be so helpful. I'll try to find people who can help too where maybe this puzzle needs a community to solve? Any feedback is welcome.

KoBeWi commented 4 months ago

I didn't look at the internals, but your gateway shouldn't be a Godot app, because it creates unnecessary overhead. Also perfectly the server shouldn't be a Godot instance either, or at least it should be as light as possible. In my project I couldn't work around that, but yours seem to be simpler though, because it doesn't really require physics. That's the advice I can give after seeing the demo.

That said, I don't really have time to provide any serious help. Seems like this is an open-source project? You'll need to find many people if you want to succeed, especially when you aren't experienced yourself, so good luck with that.

WithinAmnesia commented 4 months ago

I didn't look at the internals, but your gateway shouldn't be a Godot app, because it creates unnecessary overhead. Also perfectly the server shouldn't be a Godot instance either, or at least it should be as light as possible. In my project I couldn't work around that, but yours seem to be simpler though, because it doesn't really require physics. That's the advice I can give after seeing the demo.

That said, I don't really have time to provide any serious help. Seems like this is an open-source project? You'll need to find many people if you want to succeed, especially when you aren't experienced yourself, so good luck with that.

Okay I am back again sorry for the long reply times. Yeah the server being together is a simplicity thing. It was originally separate but it was hard to start and a lot of people struggled so it was made simpler and all together. Pros and cons you are right its better to have it separate but its more simpler to have it together.

I wonder what to do here then: "I didn't look at the internals, but your gateway shouldn't be a Godot app, because it creates unnecessary overhead." "Also perfectly the server shouldn't be a Godot instance either, or at least it should be as light as possible."

How do we get these simple yet work good together? That is a puzzle too.

"In my project I couldn't work around that, but yours seem to be simpler though, because it doesn't really require physics. That's the advice I can give after seeing the demo." You could maybe use some stuff from the JDungeon stuff its open source and like the license is really good too like how MIT works practically.

"That said, I don't really have time to provide any serious help. Seems like this is an open-source project? You'll need to find many people if you want to succeed, especially when you aren't experienced yourself, so good luck with that." yeah its going to need a open minded community for sure to solve but hey once its solved like the whole community can win and we can have this amazing community resource forever to build massive multiplayer games with. We just need a start and bit by bit we can find solutions.

You are right I'm not the bets at this but like I think if I try hard enough and we find solutions together and like it'll take a bunch of people like you say in the community to come together and solve this big puzzle and once its solved everyone benefits. Also we can stop using Unity and Unreal and other for profit game engine that could like on a whim ruin everything by being too greedy like Unity. Unreal is owned by Tencent in China so like that just one back update away from being like Unity then what? All we have Godot then and that's all we need for its open source forever free.

Once we solve this 20+ year old critical massive multiplayer player infrastructure puzzle we can start building massive multiplayer games across the entire community for the foundation would be there forever free for everyone. I'll be around just I'll try to get the community together too and rally behind this multiplayer seamless chunk loading and unloading system and try and solve this puzzle by by bit.

WithinAmnesia commented 4 months ago

https://github.com/KoBeWi/The-Soulhunter how do we get the game going? It is Godot 3.XX and the server .zip is in a strange website that won't allow the download?

KoBeWi commented 4 months ago

The link was dead, I updated it.

WithinAmnesia commented 4 months ago

Do you want to make this into Godot 4+? Your game could use a Godot 4+ update to work now? Godot is using 4+ now and its not backwards compatible. Yes there is so long term 3.XX support but 99/100 new projects are using Godot+ and the dev team is focusing on Godot 4+ and eventually wants everyone to use Godot 4+. I really hope so for there are very few really talented people like yourself.

KoBeWi commented 4 months ago

Well as I said, the project is no longer developed, so I have no interest in updating it. The source is available if someone wants to upgrade it (there isn't much code to change).

WithinAmnesia commented 4 months ago

What code needs to be updated to Godot 4 and what would that look like? What are you doing now?

WithinAmnesia commented 4 months ago

How do you get past the hue part logging into the game? i go to make an account i have both the server and client.exe running and when I type in like username 'test' password 'test' set hue to default red it like stops after pressing enter? What should be done to get the game to work? I let it go past the windows firewall too so its a mystery

KoBeWi commented 4 months ago

Did you run the database? You need to run database, then server, then start game, register account and login. I just tested and it still works.

What code needs to be updated to Godot 4 and what would that look like?

Run the project and fix all errors. Lots of stuff was renamed, so the methods need to be updated to new names etc.

What are you doing now?

My current main project is Voice of Flowers. I had big plans for The Soulhunter, but making a MMORPG is too much hassle. I might revise the project in a different form one day.

WithinAmnesia commented 4 months ago

Did you run the database? You need to run database, then server, then start game, register account and login. I just tested and it still works.

What code needs to be updated to Godot 4 and what would that look like?

Run the project and fix all errors. Lots of stuff was renamed, so the methods need to be updated to new names etc.

What are you doing now?

My current main project is Voice of Flowers. I had big plans for The Soulhunter, but making a MMORPG is too much hassle. I might revise the project in a different form one day.

"Run run_database.bat (requires mongodb installed)" in 'how to run'. is the other one Polish? I worked on Polish tank history before lol.

"I had big plans for The Soulhunter, but making a MMORPG is too much hassle. I might revise the project in a different form one day." Awwwwwwwe. You should do the MMORPG one too i know its hard but its worth it. I got booted from the JDungeon people cus the server drove them crazy they like Matthew not one more server question reee 'but how does the chunk talk to the server?' -thats it delete the questions and ban. I Asked too many questions too quickly overstressed the poor devs lol. They did there best but they having a hard time with errors too. MMORPGS are hard too. I think we are still friends with the JDungeon people its just servers are hard to expand lol. You know people like yourself are really hard to find. I try to keep track of people and treat them really good. Do you have any plans to get the MMO stuff and running? Its hard not a lot of positive stuff too or what happened last time, maybe it'll be better now? Any feedback is welcome.

KoBeWi commented 4 months ago

Yep, when you run the bat file you should see this:

image

If you don't it means mongodb is not available in your PATH.

is the other one Polish?

Yes.

WithinAmnesia commented 4 months ago

How do get the "mongodb installed"? Also I made the comment bigger above.

WithinAmnesia commented 4 months ago

Yep, when you run the bat file you should see this:

image

If you don't it means mongodb is not available in your PATH.

is the other one Polish?

Yes.

run_database.bat uh is the command prompt? there is a technical way to do this. I ran .bat files before (my memory is fuzzy).

WithinAmnesia commented 4 months ago

image okay I need the mongodb hmm

WithinAmnesia commented 4 months ago

https://www.mongodb.com/atlas/database mate you gotta buy mongodb like why use godot then? Hmmm I'll keep searching.

WithinAmnesia commented 4 months ago

MongoDB Community Server is the free and open-source version of the NoSQL database system. MongoDB Atlas is the paid and fully managed version of MongoDB that integrates natively with major cloud providers, including Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). hmmmmmm https://www.cloudzero.com/blog/mongodb-pricing/ Amazon cloud i see a lot too for dedicated servers.

KoBeWi commented 4 months ago

How do get the "mongodb installed"?

Download and install: https://www.mongodb.com/try/download/community

Do you have any plans to get the MMO stuff and running?

No, as I said, too much hassle. I spent a few months developing it and it's lagging as hell. I didn't even solve basic problems. Adding content is a nightmare when you need to consider all the networking stuff and with MMOs you want the server to be fully authoritative.

WithinAmnesia commented 4 months ago

How do get the "mongodb installed"?

Download and install: https://www.mongodb.com/try/download/community

Do you have any plans to get the MMO stuff and running?

No, as I said, too much hassle. I spent a few months developing it and it's lagging as hell. I didn't even solve basic problems. Adding content is a nightmare when you need to consider all the networking stuff and with MMOs you want the server to be fully authoritative.

The test chunk of the ARPG runs smooth?

KoBeWi commented 4 months ago

idk, I wasn't able to start the client. Although "runs smooth" does not mean it runs correctly, as it might be missing server validation and thus make it possible to cheat.

WithinAmnesia commented 4 months ago

"No, as I said, too much hassle. I spent a few months developing it and it's lagging as hell. I didn't even solve basic problems. Adding content is a nightmare when you need to consider all the networking stuff and with MMOs you want the server to be fully authoritative."

"No, as I said, too much hassle. I spent a few months developing it and it's lagging as hell. I didn't even solve basic problems." The basic problems are solved like combat, inventory, stats etc. on JDungeon and like combat works fast too? The content is somewhat easy to add as well.

"Adding content is a nightmare when you need to consider all the networking stuff and with MMOs you want the server to be fully authoritative." The content is added quite easy compared like working on WoW dev stuff from my memory. "with MMOs you want the server to be fully authoritative" what do you mean by this? Like where its secure or?

WithinAmnesia commented 4 months ago

idk, I wasn't able to start the client. Although "runs smooth" does not mean it runs correctly, as it might be missing server validation and thus make it possible to cheat.

Oh okay ummm. Hmm so wait, other people got it going on linux and not windows too. Did you open the godot editor and run 3-4 instances? I go import project, run let it import / reimport. maybe 1 st run does not work. stop, let it reimport if its being strange. wait till its reported if it wants to reimport. Onc eits happy with the proejct debug->run multiple instances-> run 3-4 instances. run in godot editor. you should see a green painting like screen with each instance with 3 choices. gate way, server, client. 1st instance click gateway, 2nd instance click server, 3rd+ instance click client. Then for the game clients click create account enter username and password confirm and it should log you into the world.

WithinAmnesia commented 4 months ago

https://youtu.be/RkikLTK98EQ?t=123 it looks like this ~2:00 mark. This is guide on how to run the JDungeon example chunk. The older ones needed more work to start but the newer one should import into godot and like at the 2:00 mark look like this and run like this too with 3-4 instances and the 3 buttons for gateway, server and client.

WithinAmnesia commented 4 months ago

idk, I wasn't able to start the client. Although "runs smooth" does not mean it runs correctly, as it might be missing server validation and thus make it possible to cheat.

Ah okay so like security features you were thinking of too that's really forward thinking too.

KoBeWi commented 4 months ago

When making any multiplayer game, the cheating potential depends on how much is determined by the client. E.g. in shooters you might trust the client a bit when it comes to character movement, so that less physics can be calculated on the server and the movement is smoother. But that allows player to use e.g. speed hacks, by modifying the movement speed in their client game. To fight that you can use cheat detection, anti-cheats etc., but I didn't even look into that. In my project I made the server fully authoritative, which means that in addition to adding game logic, I need to add a logic for syncing between server and client. This took a lot of time to even make it work, but it's very prone to lags, because the game can easily desync for various reasons, which leads to forced synchronization and character teleporting all over the place. I do have some system to make syncing easier, but it's still lots of manual work. Maybe it would be easier now that I have more experience with Godot, idk.

I managed to start JDungeon and it runs smoother than my game. Though when you see other players, their movement is weird.

WithinAmnesia commented 4 months ago

Oh WoW has some security feature that are run on the server that scan for suspicious activity and are not like hard baked into the server structure they are like 'soft' like you say the shooters have but they run like on the server. As they scan of clients that are going too fast or they have invisible triggers in the sky that detect if a person is using fly hacks and also a good mod / GM team helps this so you can have best of both where the server is less stress and the cheaters get caught. It might take a while to catch the cheaters but once they are banned they are not cheating and also the community can report people too it just need good community mod / gms to catch them.

WoW private servers have good account security but the game is 'soft' like flying and speed hacking but in the game world there are server tools that can detect cheating and also just listening to player / community helps so you can have 'soft' security but still catch 99% of cheaters. Its like instead of making the house air tight so no vermin get in you just make the path ways in easy to trap and catch all the vermin and beef that system up instead of making an an airtight system you just do the fundamentals like good account security and good server fast speeds low client latency.

Also if there are cheaters just use the various community know ways to find and catch the bad actors and warn and ban them either with automatic speed detection. Or like put up some boundaries like in wow where the devs / gms have tools to detect and then punish the cheaters. Sometimes the game bugs and its good to give temporary bans than can be appealed but you need a decent community like turtle WoW for this system to all work together smoothly.

There are botters and anti-botters and there are cheaters and anti-cheaters in the community. The community if listened to and empowered can help solve the cheating issues and it can be surprisingly effective. In OSRS and private wow server the community can be reward for detecting and help removing bots and gold farmers and other things that are 'legal' in a technical point of view but break the game in usual ways socially.

This 'soft' anti-cheat can be very strong if supported and the server can just mostly focus on making the game play experience really good with some obvious security features too but like only ~1/20 if less of players even try to cheat so if you can like 2000k people only like 100 or less (maybe even like 10-50) will even try to cheat.

Even then most of them will do so in obvious ways and people can report them. then if you have a good support community just warn them, be fair, temporary ban a few times. Then if they are really bad just ban the IP and other soft ways the community knows about too. The community can help a lot with the security patrols too it just take some trust and good game worth supporting. Just make sure the account information is safe and you should be 4/5 the way there to really good positive community server experience.

"I managed to start JDungeon and it runs smoother than my game. Though when you see other players, their movement is weird." yeah it looks a bit janky but it works good for latency but it can be 'smoothed out' with like latency compensators / little tricks but the core foundation is solid. I've seen other iterations run smoothly of JDungeon its just the web stuff is just done getting the solid networking speed achieved and combat is fast now so it should get smoothed out over time.

It has a really solid foundation though. It can run on anything and be logged into on the web it can multiplayer and have cross platform device and they ca all log into the same server all they need is to access the internet or even run from a computer its really approachable. Just tinker around with it and like it should help put some wind in your sails to help your multiplayer game projects too.

Thank you for all your thoughts and questions. Please feel free to ask more too. Can I ask you m ore question too? I'm working on so candidates for the Godot 4+ Multiplayer Seamless Chunk System. I am looking through these ~500+ Github Godot .zip projects and seeing candidates to start to build up from. So far I have a few good candidates for Seamless Chunk System. https://github.com/DennisSmuda/godot-chunking-system [2D Chunk Generator] https://github.com/dllta/ChunkLoader [2D Chunk System] https://github.com/michasng/voxel_game/tree/main [3D Voxel Generator] https://github.com/pinkavat/GroveGenChunks [3D Seamless Chunks]

So I will soon need some more testing too lol and all feedback is welcome.

KoBeWi commented 4 months ago

Well the advantage of "hard" security is that it's fully automatic. For soft security like you described you'd need many moderators; even if community reports cheaters, someone needs to go through the reports.

I'm working on so candidates for the Godot 4+ Multiplayer Seamless Chunk System. I am looking through these ~500+ Github Godot .zip projects and seeing candidates to start to build up from.

Eh, if I needed a chunk system, I'd write my own. Looking through hundreds of solutions to find a fitting one doesn't sound much more efficient, especially when the repos provide basically 0 information about the system and you need to look into the code.

WithinAmnesia commented 4 months ago

The automatic security is useful too its juts a balance of game performance and like what what really needs to be therefor security is all; every game has its own optimal balance.

The ~500+ github repos are just to make sure I cover all my bases for some of these don't have good descriptions and all it takes is some organizing and like going through them. once they are covered I can be certain of what is there and also I can find other expected help too. It's not directly efficient but its a limit amount of difficult and once its covered I can move on and also I can meet people along the way too. So its good in more than just finding what I need for code but also for people and like most people don't really use the forums that much so its good to scout the community and be proactive. I found a bunch of game type from "godot multiplayer" like 2D and 3D shooters and racing games and even uno lol. So like its not super efficient for this task but it also makes sure I check every lead and do a thorough search which may save me a lot of time if I find a very talented and just good project. JDungeon took over a year to make and I had to search for 1 day of work to find it and if I did not I might have missed it you know? Its called JDungeon not like rpg or and easy to find name.

So it takes time and a lot of patience and focus but it does work its just not easy for sure to search download every GitHub repo in a search then organize and make there own folders and unzip and then run and then document and process and then make a grading system and then do everyone one the you get short list and it really helps to find talented people from there but its a lot of focused work for sure lol.

WithinAmnesia commented 4 months ago

I am doing a from scratch build now for the multiplayer chunk system. What should I do for the server and the data and chunk storage? https://github.com/WithinAmnesia/ARPG/discussions/16

KoBeWi commented 4 months ago

I used Godot C# for the server and MongoDB for data storage. Not sure about chunks, if you want them to be persistent you should implement some custom binary serialization that only stores data you actually need stored.

There's also option to use Rust, which has Godot bindings too. But you have to learn the language first; I tried it and it wasn't for me .-.

WithinAmnesia commented 4 months ago

Update: I am doing a from scratch build now for the multiplayer chunk system. What should I do for the server and the data and chunk storage? I have a few leads and some more thoughts and resources can be found here: https://github.com/WithinAmnesia/ARPG/discussions/16 <Start fresh thread. https://github.com/WithinAmnesia/ARPG/discussions/15 <Previous start thread. It was suggested I build a from scratch Multiplayer seamless chunk system that eventually be put together where it can handle 2K-16K+ players on the same server. What should be done for a web deployment to also run on a browser? I was suggested to try and do a chunk storage system similar to Minecraft or Terraria where each chunk is stored individually and accessed by the server in communication with the player clients. What should be done to solve this chunk puzzle? All feedback is welcome.

WithinAmnesia commented 4 months ago

I used Godot C# for the server and MongoDB for data storage. Not sure about chunks, if you want them to be persistent you should implement some custom binary serialization that only stores data you actually need stored.

There's also option to use Rust, which has Godot bindings too. But you have to learn the language first; I tried it and it wasn't for me .-.

I have been on the hunt for solutions to learn from to make a ground up solution and so far I have found some promising leads for data and chunk storage for multiplayer in Godot 4.2.1.NET "Leads and Suggestions: https://github.com/Solicey/minecraft-made-with-godot/issues/1 (Multiplayer Minecraft clone written in C# with Godot 4.2.1.NET) ... https://github.com/xellu/xelcraft/issues/6 (Multiplayer Planned Minecraft clone written in C# with Godot 4.2.1.NET) ... https://github.com/hakanero/minecraft-clone/issues/1 (Minecraft clone written in C# and GDscript with Godot 4.2.1.NET) ... https://github.com/pvini07BR/mijocraft/issues/2 (GDScript Multiplayer Minecraft in Godot 3.XX) ... https://github.com/fanherbaty/crustycraft/issues/1 (2D Minecraft with cave levels in Godot 4.2.1.NET+) ... https://www.youtube.com/watch?v=LEZGrHyWNmQ + https://astruggletosurvivedevblog.blogspot.com/ (C# software engineer seamless large tile / chunk map that can possibly do multiplayer and browser support game) ... https://github.com/sirarandor/lux-terra/issues/1 (Multiplayer expanding tile / chunk game with fog of war) ... https://github.com/danilwhale/mc2d/issues/1 (2D sidescrolling Minecraft with inventory and crafting) ... https://www.reddit.com/r/godot/comments/y955vs/i_made_an_opensource_minecraft_classiccompatible/ (Minecraft classic running on Godot written in rust)"

Also data binary serialization lead: " https://gist.github.com/theraot/31515e28e2d8bfea33f6c6d5bcd852f6 (BinarySerializer for Godot 4.2+ from Theraot)"

What are your thoughts and suggestions? What should I do next? All feedback is welcome.

KoBeWi commented 4 months ago

What should be done for a web deployment to also run on a browser?

Well running stuff on a server is not much different from running it locally, just instead of localhost you use its IP to connect. Though you first need to setup your environment on the server, that means uploading your server app and starting it, and servers are usually Linux with no graphic interface (I mean, you are supposed to do it remotely, so using terminal is usually the only way).

As for hosting itself, you need a hosting service. I used Google Cloud in the past and it was fine, but of course it's not entirely free (last time I tried it had a 1-year trial version or something; though the cost depends on how many resources you need). There's also an option for self-hosting, but for that you need to setup your own server, give it an external IP address and keep it running.

WithinAmnesia commented 4 months ago

What should be done for a web deployment to also run on a browser?

Well running stuff on a server is not much different from running it locally, just instead of localhost you use its IP to connect. Though you first need to setup your environment on the server, that means uploading your server app and starting it, and servers are usually Linux with no graphic interface (I mean, you are supposed to do it remotely, so using terminal is usually the only way).

As for hosting itself, you need a hosting service. I used Google Cloud in the past and it was fine, but of course it's not entirely free (last time I tried it had a 1-year trial version or something; though the cost depends on how many resources you need). There's also an option for self-hosting, but for that you need to setup your own server, give it an external IP address and keep it running.

"What server and data suggestion should I consider too? Amazing Web Service is what JDungeon is hosted on but there are other options. It is recommended I use a fast langue like C# for server data and also GDscript can be converted to C# relatively reasonably the community says too so they get along it seems. I am trying to future proof this as much as I can to make solutions that can grow into big games.

https://www.youtube.com/watch?v=It9D08W8Z7o These big mass produced 100+ CPU miniaturized server rooms that fit in a PC are coming to consumer market in the mid to late 2020's and will be everywhere by the early 2030's as the next big computer upgrade. So I want to build that 2030's M.M.O. infrastructure today over ~2-6 years and be ready for it and aim for up to 2K to 16K+ clients on one server to try my best shot at 10,000 player battles. What are your thoughts and suggestions? What have you considered for server options in the past and going forward? All feedback is welcome." From: https://github.com/pvini07BR/mijocraft/issues/2#issuecomment-1953268020 I asked this over here but I feel like I should ask it to you as well for you seem to really have a lot of good information and deep knowledge and understanding on what would be a really good to host servers.

"Well running stuff on a server is not much different from running it locally, just instead of localhost you use its IP to connect. Though you first need to setup your environment on the server, that means uploading your server app and starting it, and servers are usually Linux with no graphic interface (I mean, you are supposed to do it remotely, so using terminal is usually the only way)." Is this how JDungeon did this but on Amazon Web Service a.k.a. AWS for ~$5-9 USD a month: https://jonathaneeckhout.itch.io/jdungeon + https://github.com/WithinAmnesia/ARPG/discussions/15

"As for hosting itself, you need a hosting service. I used Google Cloud in the past and it was fine, but of course it's not entirely free (last time I tried it had a 1-year trial version or something; though the cost depends on how many resources you need). There's also an option for self-hosting, but for that you need to setup your own server, give it an external IP address and keep it running." that is really good to hear that you have real world proven skill set and you used another service than Amazon as a reference. I asked in the past with JDungeon people about how hosting on your own PC would world and they said be careful for it s not 100% safe just takes extra time to get into so they recommended using a webserver you can wipe and make a clone of if it gets hacked and keeping all of your personal and dev stuff on separate computer / non-public networks to keep ot all safe. What are your thoughts and suggestions and or feelings on all of this web and server and security area? you seem to aha e lot of knowledge on this and alolt of 1st person experience and proven skillset too.

KoBeWi commented 4 months ago

Well I don't have that much knowledge. I just tried different stuff, but never really succeeded in the networking stuff. I did try Amazon too, but with poor results. Google Cloud was just easier to use for me. Self-hosting isn't really unsafe, granted, you should do it on an external machine, not your personal computer. Like you can have a separate PC for a server, or a Raspberry PI or whatever. Servers need good CPU and RAM, but they don't need GPU, so it's a different kind of hardware.

Also I'm not sure about using C# for a server. I've seen many people complaining about garbage collector when using C# for games, I assume it's not very scalable. Something like C++ or Rust would be better for performance, but it's more difficult to write.

WithinAmnesia commented 4 months ago

Well I don't have that much knowledge. I just tried different stuff, but never really succeeded in the networking stuff. I did try Amazon too, but with poor results. Google Cloud was just easier to use for me. Self-hosting isn't really unsafe, granted, you should do it on an external machine, not your personal computer. Like you can have a separate PC for a server, or a Raspberry PI or whatever. Servers need good CPU and RAM, but they don't need GPU, so it's a different kind of hardware.

Also I'm not sure about using C# for a server. I've seen many people complaining about garbage collector when using C# for games, I assume it's not very scalable. Something like C++ or Rust would be better for performance, but it's more difficult to write.

Well networking is hard but hey the community is getting more and more networking solutions and this is big one and its open source so once its solved everyone cane have it forever free. What was the issue with amazon? I winder why JDungeon used it strange? Maybe Google cloud is better too?

So self hosting you think is a pretty good option? I have access to a few computer I don't use and I have Starlink internet with 19-21 ms of latency and in theory unlimited bandwidth and I live a very unpopulated area so I;m not crowded for laser signal / I have a lot of internet capabilities where I am. So hopefully that can go well for self-hosting?

What do you mean by C# garbage collector, it rings a bell but i can't recall what one lol. Is 'C# garbage collector' a term or a slang or a specific thing is it like artifacts or strange like '2y271y782323r' files all over the place?

I wonder how scalable C# is for server hosting and multiplayer? I heard that GDscript is slow and C# is fast and I heard that C++ is good for servers and most servers run on Linux and I also heard C# for server side stuff done separately so I'm still wondering myself what best to do? I have seen rust on Godot there is a Minecraft multiplayer example that is written in Rust (in my links) but I'm not sure how compatible Rust is with GDscript and I see a lot of GDscript and C# getting along so maybbbeee C# server or C++ for C# I think gets along with C++ well and I hear that GDscript share a lot in common with C# so maybe they get along and like maybe the server is written most in C# or C++ and maybe the client has C# and some GDscript or something? I download hundreds of projects for Godot and lots had GDscript and C# and very few had like other coding languages I did see like C++ and Rust though too but it was more multiplayer and really uncommon but a lot of high performance multiplayer demos had more of the C# and C++ and Rust going on it feel like from memory so you probably have a good point there lol. Examples: https://www.reddit.com/r/godot/comments/y955vs/i_made_an_opensource_minecraft_classiccompatible/ (Minecraft classic running on Godot written in rust) https://gitlab.com/tpart/classy https://gitlab.com/tpart/godotcraft-classic https://gitlab.com/nokoe/classicl-server

Also https://github.com/Zylann/godot_voxel/issues/602 (Main Godot Voxel Game with multiplayer options and written in various languages with GDscript and C++ and C#) https://github.com/Zylann/voxelgame I tested this out and the multiplayer works really good but it does not save but its really good to know this exists.

The 2D Minecraft says it has seamless multiplayer chunk storage read and write with the server and client working together that already works but the save to disk / harddrive feature is not implemented / finalized but it should work the owner says. I made a thread / issue for this in hopes to get this feature some play to start. What are your experiences with save systems? What should be done to get this to work you figure? What are your suggestions? Any feedback is welcome.

There are not a lot of other options and its probably work the time to just improve this learn for it and try to use the same style of chunk storage and data management for multiplayer moving forward. It would probably be wise to get familiarized with a working Godot engine system and for to improve the same multiplayer data chunk puzzle solving from a from scratch project and custom tailor the solutions to better fit a top down 2D multiplayer seamless chunk system. What are your thoughts on this? Anything that you feel could be done or like obvious to do? What should be done to start?

KoBeWi commented 4 months ago

Maybe Google cloud is better too?

It's not like it's better, I just found it more intuitive to setup and use. Amazon is probably as capable, but I found the interface more obscure. Maybe it has changed.

So hopefully that can go well for self-hosting?

Note that for self-hosting at one point you'll need a cluster of computers. There is a limit for how much one machine can handle, especially a home PC that isn't made to be a server. Also it uses power, so hosting a server will increase your electricity bills.

What do you mean by C# garbage collector

When an object in C# is unused, the language runs a special mechanism to cleanup unused references. The problem is that they are not really tracked, so garbage collector has to find them, determine that they are unused and remove them. This causes periodic stalling while the app is running. Although there are ways to avoid that, maybe it's viable for the server. C++ or Rust don't have this problem, because they put the memory management on user (which in C++ can lead to memory leaks if you don't know what you are doing).

I'm not sure how compatible Rust is with GDscript

If it's GDExtension you shouldn't have problems with compatibility.

What are your experiences with save systems? What should be done to get this to work you figure? What are your suggestions?

The most difficult part of a save system is determining what to save. After you gather your data (e.g. in a Dictionary or Array), just do var_to_bytes() and save it. There is a WIP system for Godot that's supposed to help that: https://github.com/godotengine/godot/pull/79644

WithinAmnesia commented 4 months ago

Okay that's good to heard about the self hosting the capabilities of the cloud services. Thank you or teaching about how the C# garbage collector works and also the advantage of C++ and Rust. I wonder if 'Tuesday server reboots' like in wow help with clearing garbage or help get the memory or other long running little details things refreshed? that's good to heard that Rust and GDscript can get along more or less.

The big one is saving the multiplayer chunk data now. I have a handful of seamless chunk systems but they dont save the multiplayer data.

So far I tested and found some multiplayer seamless chunk system example leads for Godot with:

Leads and Suggestions: https://github.com/Zylann/godot_voxel/issues/602 (Main Godot Voxel Game with multiplayer options and written in various languages with GDscript and C++ and C#) https://github.com/Zylann/voxelgame ... https://github.com/pvini07BR/mijocraft/issues/2 (GDScript Multiplayer Minecraft in Godot 3.XX) https://github.com/pvini07BR/mijocraft/issues/3 https://github.com/pvini07BR/mijocraft/issues/4 ... https://github.com/Solicey/minecraft-made-with-godot/issues/1 (Multiplayer Minecraft clone written in C# with Godot 4.2.1.NET) ... https://www.reddit.com/r/godot/comments/y955vs/i_made_an_opensource_minecraft_classiccompatible/ (Minecraft classic running on Godot written in Rust and 3.XX) https://gitlab.com/tpart/classy https://gitlab.com/tpart/godotcraft-classic https://gitlab.com/nokoe/classicl-server ...

Yet they don't save the data for multiplayer.

I have yet to get your example working I get stuck in the character creation part with the hue selector. I am not sure what to do to get it working for a proper test? I would like to test it for sure its just I am not sure how to get it working?

I have some more leads I got o process like I got to get Zylann's voxel Godot 3.XX mono / .NET editor module and test a bunch of voxel / minecraft demos from github repos I could not start in 4.XX.NET. I have like ~100+ Godot voxel projects I have o look through to see if any have multiplayer and save features but like I already found the best ones with other searches so its unlikely but its an exhaustive search. I just need multiplayer seamless chunk system that can save the chunk data and work with the multiplayer server like how Minecraft does it but for Godot this time. I should learn the high like multiplayer API saving more it looks very promising. There is a little prototype demo here too: https://github.com/godotengine/godot-demo-projects/pull/935