How should events be defined ?

[ddubbert]

There are two possibilities:

1. Send the events for each user by itself. This is a more fine granular and flexible approach but might need more processing effort. Processing services will need to hold state to be able to compare game object positions and detect collisions. Scaling with Kafka Consumer Groups is harder, because events depend on others and services would need to be synchronised.

2. Bundle the events for each user into a bigger structure. Events are bigger and might need more transfer time but services are allowed to be stateless, because all the data they need is bundled and served by the event. In addition, Kafka Consumer Groups can be used for scaling without the need of any synchronisation between services for the same task.

The speed of those approaches is the dominant question in such a realtime environment.The game should at least be able to provide framerate of 30fps (but 60fps would be better). A Proof of Concept should be made for testing each approach before making a decision.

[ddubbert]

Different PoCs have been created and their time for sending and receiving 30000 events with different sizes have been measured (single and grouped events, groups consisting of 1000 events). The amount of 30000 was chosen because it represent the events needed for displaying 1000 users with 30fps. One try also has been with 60000 grouped events (grouped to messages of 1000) to test for 60fps. For all of them an average value has been calculated out of 5 tries. The events are also serialized / compressed with the help of avro-schemas. The sizes before and after compressing are also listed.

Single Events, 30.000, 38 Bytes / 8 Bytes: (Broker 1 & Partitions 3 & Replica 1)

• Acknowledgment -1 (all Partitions):

  • Milliseconds: 9234, 9126, 9039, 8957, 8971
  • Average: 9065.4

• Acknowledgment 1 (only leader Partition):

  • Milliseconds: 8861, 8988, 8823, 8991, 8813
  • Average: 8895.2

• Acknowledgment 0 (none):

  • Milliseconds: 3943, 3761, 3651, 3541, 3406
  • Average: 3660.4

Grouped Events, 30.000, 38018 Bytes / 8003 Bytes: (Broker 1 & Partitions 3 & Replica 1)

• Acknowledgment -1 (all Partitions):

  • Milliseconds: 84, 108, 106, 105, 92
  • Average: 99

• Acknowledgment 1 (only leader Partition):

  • Milliseconds: 89, 112, 103, 81, 84
  • Average: 93.8

• Acknowledgment 0 (none):

  • Milliseconds: 71, 81, 77, 67, 75
  • Average: 74.2

Grouped Events, 60.000, 38018 Bytes / 8003 Bytes: (Broker 1 & Partitions 3 & Replica 1)

• Acknowledgment -1 (all Partitions):

  • Milliseconds: 133, 193, 196, 178, 143 (without console.log: 130, 132, 108, 106, 100)
  • Average: 168.6 (without console.log: 115.2)

• Acknowledgment 1 (only leader Partition):

  • Milliseconds: 126, 185, 169, 133, 166 (without console.log: 98, 95, 85, 102, 94)
  • Average: 155.8 (without console.log: 94.8)

• Acknowledgment 0 (none):

  • Milliseconds: 114, 182, 155, 150, 153 (without console.log: 46, 47, 38, 44, 50)
  • Average: 150.8 (without console.log: 45)

Grouped Events, varying Sizes, try to keep 60 messages per second: (Broker 1 & Partitions 3 & Replica 1)

Tested by multiplying following data structure several (thousand) times:

{ id: 100, x: 100, y: 150 }

• 1000 times, 8003 Bytes: 98, 56, 51
• 10000 times, 80004 Bytes: 102, 76, 68
• 50000 times, 400004 Bytes: 478, 532, 535
• 60000 times, 480004 Bytes: 512, 518, 551
• 65000 times, 520004 Bytes: 552, 584, 532
• 70000 times, 560004 Bytes: 902, 968, 987
• 75000 times, 600004 Bytes: 901, 1029, 979
• 80000 times, 640004 Bytes: 1172, 1182, 1136
• 800004 Bytes: 1229, 1175, 1257

[ddubbert]

It seems, that the size of a message hasn't got as much impact on the performance as the amount of messages send and the time needed for each transaction. Having bigger messages results in less messages which will improve the performance factor. Whenever it is reasonable to group events, it should be done. In addition an acknowledgment factor of "none" will be used. Loosing single events in this context is worse than loosing all events of a frame because users could be favored while others would be neglected (e.g. users might move while others wont). Lastly the average values of grouped events show that message-wise a frame rate of much more than 60 fps would be possible. The bottleneck will be processing all of these events and rendering the game.

Yet the size should be considered. Even though the program wont need to send multiple thousands of nodes per event (will only be like 100 - 1000 nodes on the board), the size of each node will be higher than the data structure used for testing. A maximum size of something between 400004 Bytes (0.400004 MB) and 520004 Bytes (0.520004 MB) should be targeted to compensate for multiple Topics, parallelism and latency.