Socket.io API endpoint for players of a game to start real-time communication

[metalalive]

API endpoint to socket.io server

• localhost:8082 , path / ,GET method, no request body
• Identity authentication and authorization (e.g. [verify JWT signature) will be postponed
  • we can do that when we start JWT implementation for HTTP RESTful server

Interaction with socket.io server

Client (Player)

Possible events namespace to emit (or topics to publish)

• /game/{game-id}/join <-- Not sure whether this is good design option

  • For later comer who attempts to join a game which hasn't been started yet
  • Possible arguments : e.g. player-id = abc999xyz
  • Note we have not decided which binary format to apply to event message, but it is not JSON for sure.

• /game/{game-id}/chat

  • For player chatting with other players of the same existing game
  • Possible arguments : e.g. player-id = abg007xyz, chat-message = Hello World

Possible events namespace to receive (topics to subscribe)

• /game/{game-id}/config/difficulty
  • refresh game difficulty
  • Possible arguments : e.g. level = standard,
• /game/{game-id}/config/investigators
  • refresh the config, shows which player successfully selected which investigator
  • Possible arguments : e.g. player-id = SamuraiBuck, investigator = magician

There should be many more topics to subscribe for each player

• especially when the game started , each player completes any action or draw any card in his / her turn.
• for now , I simply list a few we can try to implement in the beginning.

HTTP RESTful server

Possible events namespace to emit (or topics to publish)

• /game/{game-id}/config/difficulty
• /game/{game-id}/config/investigators

Possible events namespace to receive (topics to subscribe)

• None

Question

• event namespace /game/{game-id}/join , <-- is this good design option ?
  • what about adding new REST endpoint /game/{game-id}/join ? to let the HTTP RESTful server know one more person would like to join , then re-use Game.add_player() in backend code.

TODO

• implement minimal working socket.io server
• infra setup (python-socketio and Kafka)

Reference

See previous discussion #29

@wingtkw @teds-lin , plz take a look at this, feedback or suggestion are welcome , thanks.

[teds-lin]

• /game/{game-id}/join <-- Not sure whether this is good design option
  • For later comer who attempts to join a game which hasn't been started yet

設計方針是只與GaaS大平台串接還是會有考慮其他遊戲登錄平台？ 這是指之後有其他玩家也想要加入遊戲房間，所以一同遊玩人數會變更的情境嗎？ 如果是GaaS大平台串接， 因為是等大平台那邊玩家都組好隊， 才會送過來，當送過來後， GaaS也沒辦法在新增玩家進來， 所以我認為先不考慮這個功能

• /game/{game-id}/chat

不是很懂，看前面一條我以為是在討論REST endpoint? 可是下面解釋說是"player chatting with other players" 所以是在講socket.io? 我不認為玩家聊天API需要在REST endpoint也建置對應的，徒增複雜度 socket.io的話 namespace 官網圖有點難懂 其實就是像這篇文章介紹的 Client可以建立很多socket連線到不同的namespace，每個namespace又可以分很多room

在我們遊戲這個case， 我認為很簡單的用原本預設的/就好， 然後共同遊戲的玩家都在同一個game-id的room內

There should be many more topics to subscribe for each player

是在講Kafka?我還沒開始看，暫時不了解設計上需要什麼

如果是在講socket.io的event namespace 目前我想到兩個設計方案 (1) 因為是傳送和game相關的可能是命名為game_state或者直接叫game之類的 通常event namespace和上面剛剛提到的namespace命名上最大區別就是最前頭會不會加上/ 如果是與玩家聊天相關的內容， 傳送資料結構內可能會用Message_Type來區分兩者。 (2) 另外一個設計方案就是 將玩家聊天相關內容獨立出來，event namespace命名就chat之類的名稱 這樣可以明確分隔傳送的是遊戲狀態更新(game)還是聊天訊息(chat)

加上之前討論我說

考慮只傳送差異更新而不是完整遊戲狀態

為了舉例方便，我用JSON示範 具體程式碼片段可能會像

# 已從Kafka接收到新的難度等級
new_difficulty_level = STANDARD

# 發布遊戲狀態更新到前端
sio.emit(
    'game', 
    {'event': CHANGE_DIFFICULTY_STATE,
      'difficulty': new_difficulty_level},
    to=game_id
)

[metalalive]

Hi, Ted, 我先回答部分問題

設計方針是只與GaaS大平台串接還是會有考慮其他遊戲登錄平台？

只考慮 GaaS 大平台

這是指之後有其他玩家也想要加入遊戲房間，所以一同遊玩人數會變更的情境嗎？ 如果是GaaS大平台串接， 因為是等大平台那邊玩家都組好隊， 才會送過來，當送過來後， GaaS也沒辦法在新增玩家進來， 所以我認為先不考慮這個功能

是的, 目前的 GaaS 大平台 無法增加更多玩家到 已存在的遊戲房間 我以為這部分是由每個遊戲應用程式處理, 所以我最初的問題是 :

• 我們是否應該添加 REST endpoint /game/{game-id}/join , 讓一些稍後過來的玩家 能進入房間 ?
  • 例如，一開始有2個人，然後遊戲開始之前 ,又有第三位玩家 想加入
• 或者，讓玩家先連接到socket.io server , 然後 subscribe event , with the namespace /game/{game-id}/join ?

但ok , 這也不是關鍵功能，可以先不考慮

---

• /game/{game-id}/chat

不是很懂，看前面一條我以為是在討論REST endpoint? 可是下面解釋說是"player chatting with other players" 所以是在講socket.io?

這不是 REST endpoint, 這是 event namespace , 這是官方文件中的範例

• 假設 玩家 (client) 透過 https://localhost:8082/ 連接到 socket.io server
• 連線建立後, 玩家 可以決定監聽哪個 event , with custom namespace such as :
  • /game/{game-id}/chat , 接收同一遊戲房間 其他玩家的訊息
  • /game/{game-id}/config/investigators 如果任何玩家更改了 investigator ，這個事件應該反映到同一房間的其他玩家
  • /game/{game-id}/config/difficulty , 如果任何玩家更改了 difficulty level ，這個事件應該反映到同一房間的其他玩家

所以我最初的comment 是問你們對 namespaces, topics 的命名規則有何偏好 與 REST endpoint 無關，抱歉造成混淆

---

其實就是像這篇文章介紹的 Client可以建立很多socket連線到不同的namespace，每個namespace又可以分很多room

OK

• 簡單的 topics (to subscribe) -->
  • /game for 遊戲狀態更新
  • /chat for 聊天訊息
  • /something/else
• 比較複雜的 topics -->
  • /game/{game-id}/chat 如果您希望聊天訊息僅在 特定房間的玩家可以看到
  • /game/{game-id}/config/investigators
  • /game/{game-id}/state/card-draw
  • , 其中 {game-id} 是代表遊戲 ID 的動態字串
  • under the namespace game/{game-id} , there could be rooms such as chat, config , state ...etc
    • does it match the concept of rooms ?

在我們遊戲這個case， 我認為很簡單的用原本預設的 / 就好， 然後共同遊戲的玩家都在同一個game-id的room內

Where to place the game ID ? should game ID be part of topic or namespace ? like /game/{game-id}. Or each room under the namespace /game is identified by game ID ?

[metalalive]

Hi, Ted , continue to reply rest of the questions

是在講Kafka?我還沒開始看，暫時不了解設計上需要什麼

please ignore the message , it is unrelated to Kafka , currently let's focus on socket.io namespaces in the game app.

如果是在講socket.io的event namespace 目前我想到兩個設計方案 (1) ...... omitted ........ (2) 另外一個設計方案就是 將玩家聊天相關內容獨立出來，event namespace命名就chat之類的名稱 這樣可以明確分隔傳送的是遊戲狀態更新(game)還是聊天訊息(chat)

I prefer the 2nd design option, but I am not sure whether I can set up the namespace such like /game/{game-id} in the backend app (where {game-id} is a variable can be changed dynamically) , I will try the library python-socketio in my local development environment & reply later with my experiment result .

[wingtkw]

想問一下 State card draw 是想用來做甚麽的 是回合抽卡完成 誰的回合 還是別的

我在想把卡有關的的endpoints 可以group 到 /game/{gameId}/card 之下 所以想了解一下這條endpoint 的用意是甚麽

[metalalive]

State card draw 是想用來做甚麽的 是回合抽卡完成 誰的回合 還是別的

抱歉，我只是舉個例子, 目前我還沒考慮實作細節 請忽略這個 card draw

Finally I got some time to work on the socket.io server, I will first start implementation which publish / subscribes message with the following topics -- 我目前正在做的 socket.io topics

• /game/{game-id}/chat
• /game/{game-id}/config/investigators

[metalalive]

To everyone following / interested with this issue, here is the experiment result, feel free to ask question, I appreciate any feedback about the code and design, If I don't get any reply by the end of Thursday afternoon (17:30:00 , 7th. Dec. 2023) , I will continue working on the real-time chat implementation and then quickly create a PR.

---

[1] The comment in this python-socketio issue describes that socket.io client with http requires aiohttp , otherwise there will be timeout exception on the client side. In this game backend app , socket.io client is used ONLY for testing purpose, it is suggested to install aiohttp in test dependency of poetry config file.

[2] the namespace in python-socketio seems more like a constant than a variable, it might NOT be good practice to generate namespace dynamically at runtime. Quote my previous comment :

I prefer the 2nd design option, but I am not sure whether I can set up the namespace such like /game/{game-id} in the backend app (where {game-id} is a variable) , I will try the library python-socketio in my local development environment & reply later with my experiment result .

After coding with python-socketio library, I decided to give up doing that , instead I set {game-id} to room with each event and namespace .

[3] For testing

• Initially, I tried AsyncClient , which monitors events by registering event-handling callbacks to the method on(...) , however the handler callback does not seem to provide any argument that represents which client is receiving which message , which is NOT friendly to tests.
• Now I use AsyncSimpleClient in test case, it provides the method receive(...) for getting events from socket.io server, this is good for understanding which client is receiving which message.

[4] I pushed my commits to the branch experiment/backend/socketio (it is only for experiment purpose) , this comment will be no longer edited .

[metalalive]

To @wingtkw : I have done the socket.io experiment (see the result above)

I am considering to add another API doc at doc/api for the socket.io real-time communication, this document requires both of frontend and backend contributors working together, can you check the test case in the branch /experiment/backend/socketio to see whether is it OK for chat event ? currently there are fields like client ,gameID , and msg in the chat event, please take a look , thanks.

To anyone interested with the work, you are also welcome to discuss with us , (I don't tag anyone else here, feel free to get involved), thanks

[metalalive]

TODO

the list is what I will do :

Portion 1

• replace print message with logger in socket.io server
• add fields like nickname to the data of the chat event (what other fields required by frontend app ?)

• remove first forward slash character / from each event name, e.g. /chat renamed to chat

  create PR, merge to main branch --- (done in 11th Dec 2023)

Portion 2

• For detecting re-entrants , as well as the players who have never joined a game room
  • socket.io server needs to know which game rooms are created, which players are currently in which room
  • feasible implementation , how can socket.io server knows about it ? :
  • socket.io server maintains some kind of table map, and the table map records events emitted from REST server on creating game <-- (preferred, more de-coupled)
  • socket.io server directly read from game repository through the abstract interface.

Remind that :

• GaaS Lobby platform creates a new game , by sending request to the REST endpoint /games , with POST method and list of player IDs
• currently we (this game dev team) don't consider to let more players (or late comers) join an existing game room

create PR, merge to main branch --- (done in 26th Jan 2024)

Portion 3

• REST HTTP server sends event for updating character selection and difficulty level of specific game, to socket.io server, the socket.io sevrer forwards the events to the players

create PR, merge to main branch --- (done in 1st Feb 2024)

Portion 4

• for binary payload format , try switching to FlatBuffers or ProtoBuf (see previous discussion)

create PR, merge to main branch --- (done in 7th Feb 2024)

Tasks NOT included in this TODO list

• ensure only HTTP RESTful server has privilege of emitting game-state-update events to socket.io server , it might be related to Kafka task below.
• Kafka task is moved to the issue #56 , I need to figure out the appropriate usage context before applying Kafka.

[teds-lin]

通常event namespace和上面剛剛提到的namespace命名上最大區別就是最前頭會不會加上/

我注意到實驗分支 experiment/backend/socketio 實作中 event namespace 命名開頭都加上/, 例如 /chat, /init等 好奇這樣取名原因是？

一些相關討論： Custom event naming convention Is there any name convention for response event in socketIO?

補充說明： namespace會當作URL的一部分，所以開頭第一個字元固定為/ 如果預設連接的URL為http://xxx.example.com/ 用預設的namespace="/" 當為socket.io時，URL實際連結會像http://xxx.example.com/socket.io/?EIO=4&transport=websocket 當namespace="/game" 實際上代表連接的URL為http://xxx.example.com/game 當為socket.io時，URL實際連結會像http://xxx.example.com/game/socket.io/?EIO=4&transport=websocket 至於event namespace 是socket.io內部通訊event實作，跟連接的URL無關

I am considering to add another API doc at doc/api for the socket.io real-time communication,

socket.io 文件通常用 AsyncAPI 撰寫, 但我不確定同樣放在doc/api這樣能不能相容, 或者放在doc/socketio？ Pt1 Pt2

[metalalive]

Hi , Ted

event namespace 命名開頭都加上/, 例如 /chat, /init等 好奇這樣取名原因是？

The naming convention of socket.io event really depends on development team (at least the name has to reflect the corresponding operation done by server or each client), it can be changed. And yes, there is no reason to add extra slash symbol / at the beginning of event name, this can also be removed, current events can be renamed to chat , init, deinit

socket.io 文件通常用 AsyncAPI 撰寫, 但我不確定同樣放在doc/api這樣能不能相容, 或者放在doc/socketio？

thanks, good to know about AsyncAPI and event-driven architecture API document It is also OK to place it in different path such as doc/socketio ,

[metalalive]

Let us focus on the 2nd portion of the TODO list in this GitHub Issue , discussion related to the design / implementation are appreciated.

• the REST server emits event on creating new game
• then socket.io server receives the create-game events to know which players should be in which game room.
  • there will be adapter interface to data store, which simply records currently active players in which rooms
  • the abstract class name hasn't been determined yet, maybe something called RtcRoomRepository or what else.
  • the dev team can switch between different storages e.g. in-memory data store or non-relational database if you like.
  • for simplicity, currently the socket.io server stores the events to in-memory data store

[update 15th Dec 2023] I am fighting against tricky tech issues in other projects, will start working on this once I have time.

[update 11th Jan 2024] I'll also check whether I can get this done without setting up Kafka

[update 23th Jan 2024] http server can act as socketio client emitting events to socketio server without Kafka

• currently there is only one http server, not a real distributed environment
• I will figure out how it works internally in the future , reference doc

[wingtkw]

Let me review it on 15 Got AL on that day

[metalalive]

I am working on the 3rd portion of the TODO list in this GitHub Issue , discussion related to the design / implementation are appreciated.

[metalalive]

I will work on the 4th (final) portion of the TODO list in this GitHub Issue.

I'll try FlatBuffers first with socket.io server and see how it goes

@wingtkw , @KalpasWang , @liamlin1102 Appreciate discussion / your feedback related to the design / implementation.

[metalalive]

Currently close this Issue, any code / design contributor can reopen it whenever there is new event to add in the real-time communication.

c.c. @wingtkw , @KalpasWang

[KalpasWang]

@metalalive it seems everyone disappear except us, what happened? 🤔