Feature request: multiplayer game with server API and more features

[CrsiX]

Hi, we really like the game but the multiplayer functionality just sucks, sorry. (And the server is pretty ... hacky. Maybe a plain FTP server would have been a better choice).

Therefore, we would like to put our hands on a new multiplayer service. We essentially build our own server (in Rust) while also hooking the relevant client functionality. Additionally, we integrate feature requests such as the following:

• 8476 (yes)

• 8063 (definitively)

• 7745 (pretty far ahead the roadmap, but a nice idea)

• Also, we want to completely overhaul the API, like using WebSockets/HTTP2 instead of polling on intervals and much more intuitive user experience.

However, we would like some help on integrating some of the stuff in the client app. For example, we build the server and the API and all of the network-related code, while someone else could jump in to tweak the user interface to make use of the newly available features (e.g. chatting or username/password instead of UUIDs).

Side question: how many people are currently using your public multiplayer service? A rough approximation (20 a day or 20k a day)?

[CrsiX]

I was just throwing ideas into the ring.

Just as always :) I'll start with that later using a new key. And the extra flag could be avoided by allowing null as well:

1. If the setting is null: the user didn't consent to anything, don't use anything.
2. If the setting is empty: the user consented, but didn't use anything yet -> this would now be the perfect time to ask for a preferred server or to give a list of available servers to select from.
3. The setting has some values: use them. However, I would add a key that indicates the currently enabled setting, so that using multiple home servers in parallel becomes easier (or possible at all).

... and by installing Discord you pretty much consented to a loss of privacy of inscrutable extent - I mean what other Software has the chuzpah to install a protocol handler, ugh that's not much better than a rootkit........ Pardon me.

Loosing privacy was no reason people use ~crappy messengers phoning home~, pardon, Discord nowadays?

[touhidurrr]

@CrsiX and @myOmikron, just curious about the status of the project. When are you guys going to launch it??

[CrsiX]

Overall, the most important parts are fully working.

For the backend side, we're on a good track. A few little things need to be fixed and extended, but it's stable overall. For the client, I'm working on the UI the last weeks. The core components (lobbies, friends, chats, game handling) are working, but they deserve further integration into the game and the various screens. A few things are not working yet (especially the Android turn checker), but I think this can be addressed after bugs and UI improvements are there.

I'm not sure what you mean by "to launch it". The backend is publicly running for you to try (https://runciv.hopfenspace.org). The client is forked here, where I already did a debug build for you to try a few days ago, if you want to.

So, I will start a PR to this repo after those things on the client are fixed. We can discuss the implementation-related details there, too. However, I wouldn't call it launching yet, since other important features like internationalization are not addressed yet.

[touhidurrr]

Also, did you prefix everything with /api? So, will your server address be https://runciv.hopfenspace.org or https://runciv.hopfenspace.org/api?

Also, where is /isalive?

[touhidurrr]

Also this is too confusing... I am not sure how this will work. Currently Unciv assigns a random UUID to the client upon installation. Now it looks like you are preserving both UUID and username. Now the confusing part is what happens when the user signs up, uninstalls the client, and then tries to log in. What will be the UUID in this case? Will it be the old one or the new one? And if the user has to set a username in the first place then why do we need a UUID? Also what happens to the games that are currently being played? They also have the user ids of the players.

I am a bit confused about how this works, can you kindly explain?

[CrsiX]

Also, did you prefix everything with /api? So, will your server address be https://runciv.hopfenspace.org or https://runciv.hopfenspace.org/api?

We prefixed everything with /api. The reason for that prefix is to easily allow the use of / and other non-API endpoints, which allow a web page or something similar to be served there. Of course, that may be changed (e.g. strip the /api from the API definition but set the baseUrl to include /api).

Also, where is /isalive?

At first, we have decided against such an endpoint:

1. If you want to check if a server is reachable, just call some arbitrary endpoint. If you receive a response (even if it may be 401 or 404), the server is reachable. A dedicated endpoint which just returns true is not helpful.
2. The /api/version endpoint fits into the rest of our API schema, since its no versioned endpoint, i.e. the purpose is to determine the version of the running backend. In our case, the response is therefore {"version": 2}.

Of course, the server can be changed. So, if /isalive is an important endpoint, we could just integrate it.

Note: in the current client, I'm checking the /isalive endpoint to determine if it's an old APIv1 implementation, because all of them return a 200 response. But since we didn't define that endpoint, it's 400 in the new APIv2. That would need to be changed, too.

[CrsiX]

I am a bit confused about how this works, can you kindly explain?

Sure. So, why a user UUID in the first place? Well, we want to uniquely identify an account. All data associated with an account may change (username, display name, password, games, whatever ...), so that UUID is the anchor to identify the user across the account's lifetime. (The UUID is also the primary key in the database.)

How do you map from a username to a UUID? Either you use the dedicated endpoint lookup or you login with username and password and use GET /accounts/me to receive your own UUID. Since it won't change, you can safely store it on the client (until logout or uninstall).

Currently Unciv assigns a random UUID to the client upon installation. Now it looks like you are preserving both UUID and username.

We can't let the client decide which UUID it will become on the server, that would just be completely broken. As lined out above, the username (plus password) is used to login. It doesn't matter if it's a new installation, another device, another timeline. The successful authentication provides the UUID. It will then overwrite the UUID of the client. (Yes, this means going back to the old API is currently not possible after first use of the new API. I have a warning popup before the first login which displays the old UUID of the player, so that it's possible to manually reset the UUID.)

Also what happens to the games that are currently being played? They also have the user ids of the players.

There's currently no plan of an "easy" transition from APIv1 to APIv2. You stop/complete all APIv1 games and then start with APIv2. Why that? Because a) we can't let the user decide on any UUIDs and b) it's just a PoC for now. Those transitions can be checked later on.

However, playing with APIv2-players only works of course. So if we have player A and player B both registered to our server, they can play the game happily together, because their UUIDs match what's in the game files. (I have tested up to four concurrent players, so I assume it's working with even more reliably.)

[touhidurrr]

There's currently no plan of an "easy" transition from APIv1 to APIv2. You stop/complete all APIv1 games and then start with APIv2. Why that? Because a) we can't let the user decide on any UUIDs and b) it's just a PoC for now. Those transitions can be checked later on.

So, it boils down to me needing to make a tool for safe transition of UUIDs. Noted.

[CrsiX]

So, it boils down to me needing to make a tool for safe transition of UUIDs. Noted.

I'm not sure about that yet. It would mean that there's a transition where all "old" user IDs are mapped to "new" user IDs at once. It may be possible, but it's not very easy. Therefore, I suggest to keep all those transitioning stuff on hold for now :)

[touhidurrr]

Also, @CrsiX can you add a flags parameter to users? like { flags: ["admin", "moderator" ] }

[CrsiX]

Also, @CrsiX can you add a flags parameter to users? like { flags: ["admin", "moderator" ] }

What exactly do you mean? What are those flags possibly used for? I assume you want something like different privilege levels, so which feature would need privileges in the first place? We have no privilege system at the moment (except for lobbies, where the owner of the lobby can do more than the members of the lobby, like kicking users and starting the game). Could you please give some examples for the usage of such flags?

[touhidurrr]

They would just appear as badges next to user. A cool thing.

[yairm210]

Let's get something working end to end for users before we start expanding into "nice to have"s :)

[CrsiX]

They would just appear as badges next to user. A cool thing.

But what's the purpose of such a badge? And how would you earn it, who decides which user gets such a badge? Things like moderator or admin need the server operator or an entity with comparable "power" to give & take those badges. IMHO, this is much more complex than it could be useful. If the user could change / set his own badge, then it would be rendered useless (or, purely aesthetically, with no real meaning for the game).

[touhidurrr]

But what's the purpose of such a badge? And how would you earn it, who decides which user gets such a badge? Things like moderator or admin need the server operator or an entity with comparable "power" to give & take those badges. IMHO, this is much more complex than it could be useful. If the user could change/set his own badge, then it would be rendered useless (or, purely aesthetically, with no real meaning for the game).

With this things like Server messages and Server bots would be possible. With verified badges. Also, people often asks me to do stuff like fix broken games, delete games, change user ids, etc. So, I want an admon badge. . Maybe the server can even give exp and levels to users. Things like this make the game more engaging.

Anyways, let's launch a working v2 first and then we can maybe add these features.

[CrsiX]

With this things like Server messages and Server bots would be possible. With verified badges. Also, people often asks me to do stuff like fix broken games, delete games, change user ids, etc. So, I want an admon badge. . Maybe the server can even give exp and levels to users. Things like this make the game more engaging.

Thanks for clarification, I now understand what you meant. Some things are really nice ideas, indeed. :)

[CrsiX]

I'm planning on introducing persistence later on. Part of this persistence will be the list of games and their game states, as well as the game chats. Since my data formats need to be different than all currently available data formats in the game, I wanted to introduce a new directory that stores the data specific to those new game mechanics. At the moment, the game saves the following files and directories (afaict):

• GameSettings.json
• MultiplayerGames
• SaveFiles

---

I want to add a new directory MultiplayerGamesV2 (working name, may get another name?) and some sub-directories that allows me to organize details related to new multiplayer games. It would then provide the following files/dirs:

• MultiplayerGamesV2/Games.json for a list of the currently played games as well as some details about them, for example
  • last player
  • last activity
  • data ID
  • game ID
  • game name
  • multiplayer server the game is played on
  • may also include the details currently available in GameInfoPreview
  • file version
• MultiplayerGamesV2/Games/{some-uuid}.json for the full GameInfo of a save game, just as the old format in SaveFiles -- all extra meta information should be placed in the above Games.json file; the different storage directory is important to prevent issues with APIv1 games
• MultiplayerGamesV2/Chats.json for a list game chats and its meta information
  • chat rooms
  • multiplayer server the chat room is hosted on
  • members
  • UUID
  • file version
• MultiplayerGamesV2/Chats/{some-uuid}.json for the full history of an active chat room (lobbies may not be stored there, since they are usually not so important)
  • we have the scenario of a really long-running game with a few thousand chat messages in mind, therefore splitting the single chat rooms into their own files
  • the chat rooms should be cleared when the game has been completed, of course

All of those new persistent data should always be

• backward-compatible and future-extensible (hence the file version)
• contain the detail which multiplayer server they were used with, so that moving between them is easy

[SomeTroglodyte]

That sounds more like a job for sql(ite)

[CrsiX]

That sounds more like a job for sql(ite)

Fair. We already discussed it internally. And it is the better alternative, indeed. However, since I didn't dive into Kotlin/SQLite/Android integrations yet, I just came up with the good old JSON. And wouldn't it require yet another dependency? I remember that Android provides some SQLite db for applications, does it? And it would require cross-platform support, of course. Quick searching brings up Exposed as an ORM, for example. I could start by evaluating the support of those/alternatives to integrate them properly into the game, if you don't have something in favor?

[CrsiX]

By the way, do you want to check out the current state of our implementation? I've just created an alpha release here. 🎉

You can freely test the new features and user interfaces that I've created there. I'm always open for suggestions and improvements. Just make sure to open any issues related to the new mechanics at our fork. Of course, you can also check out our dev branch and build it for yourself. Thank you very much :)

[SomeTroglodyte]

alpha release here.

Uh, I don't do multiplayer. But thanks for the confidence. I can confirm that checking out your branch builds and runs perfectly fine - and I see no difference, that's how good I know the MP side of Unciv. Besides, I redirect server to a not-open port on localhost to silence all that chatter (& less stutter in single-player gaming). :shrug:

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[CrsiX]

Dear GitHub Actions bot, even though I was inactive, I currently still plan to bring this feature into existence at some point in the future :) That said, I don't know when this will be >.<