[Discussion] Tackling performance issues and Loadbalancing

[mike-code]

Now that XMage gets more and more popular averaging 900 users in peak, the responsiveness of the user interface is heavily affected. You can observe that by reading chat and playing yourself.

I have not looked into the network protocol part of the code so I'll leave that to people who designed it to make a comment on that if it could be even more compact (unless it's not the bandwith that's the issue). Or perahps too heavily compressed and it's the decompression on the server side that eats up so much computing power? In such case perhaps we can consider removing it? XMage is purely desktop game so it doesn't require to be bandwith efficient on the client side (we're not targeting mobile after all).

Or is it the AI that's so heavy? Perhaps consider then moving AI to another server and host xmage servers that are AI only.

Last but not least (and probably most important) -- I understand that XMage can't work under loadbalancer (under current implementation) -- that is -- you can't run multiple instances of the same server on multiple machines and have only 1 of them hosting "lobby" which only shows list of users and running games but let the games be handled by different servers?

I'd like to leave it open to discussion what others can do because hosting additional servers doesn't make much difference if they are going to be empty then nobody will use them anyway.

[LevelX2]

It's no CPU problem.

The network code is based on old JBoss frame and is probably a bit buggy. Also the needed bandwith of the data transfered is very high.

So more bandwith would help (it's limited from the server provider). It was less limited before, so >900 users did work much better. As you see it's limited to ~120 Mbit now.

Possible improvements:

• More server bandwith.
• New/better network code by using another network framwork (e.g. Netty)
• Less data transfer from server to client (by using compression or more intelligent data handling needing less bandwith.

All this points would help to improve the situation.

[derlhurgoyf]

I actually like the idea of a lobby where games area distributed among the linked servers. Not sure how much work this would mean on the code base, but it would certainly allow more people to simply link their servers to the lobby to contribute and reduce the load. Currently the US server has the best intentions, but no users.

[theelk801]

How difficult would it be to implement something like Netty? I was looking around and found this but I don't really know how the netcode actually works so I have no idea how to implement it.

[mike-code]

Wow that's a lot. It means that each players receives in average 20KB of compressed (!!) data each second in a turn based game. It also mean that you eat up at least 30TB of transfer per month. It's actually good for us that alfahosting has flat rate for transfer otherwise it'd cost a lot.

About optimizing network code, it must be done sooner or later because even if we increase the network cap now, it will keep growing until it reaches limit that will be beyond financial capabilities of a non-profit project. But you know what the say, developers cost a lot, machines are cheap. Of course on the long run its bullshit.

However, I did some research on the internet for reliable providers with larger unmetered bandwith but I don't know how much resource is xmage using (cpu, ssd and ram). Check https://www.scaleway.com/baremetal-cloud-servers/ and https://blazingfast.io/vps

[Zerrisx]

Has any work/consideration been given to this? Seems worth checking out.

[JayDi85]

The network engine still needs to be reworked.

The simple way and the greatest improve in performance are incremental data transfer and consistency check instead full data transfer in current version.

[LevelX2]

There still does exist the old branch with the network update (done with netty by @BetaSteward ) that was (nearly) finished. I also did release it to the xmage.de server but had to revoke it because of massive performance problems. Probably it needs some fine tuning.

But as a starting point for someone who wants to know how to switch/update to netty maybe a very good source of knowledge.

https://github.com/magefree/mage/tree/Network_Upgrade (my copy with some more fixes)

https://github.com/magefree/mage/commits/Network_Upgrade (the original branch of BetaSteward)