This repository contains a solution for hosting persistent world games and virtual worlds on AWS leveraging Amazon GameLift, a fully managed game server hosting solution, with a serverless backend service and management layer. The solution is designed for getting quickly started with persistent virtual world game or application development on MacOS and Windows. It includes infrastructure as code automation, as well as deployment scripts to deploy all the required resources, and includes a sample Unity client and server implementation.
This Readme includes the architecture overview, as well as deployment instructions and documentation for the serverless backend services as well as the Amazon GameLift infrastructure.
Note: “The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.”
The project is a simple "game" where any number of players (based on your world configuration, up to 200) will join game worlds, move around them, and their location will be persisted in Amazon DynamoDB for future sessions as an example of data persistence. The movement inputs are sent to the server which runs the game simulation on a headless Unity process and syncs state back to all players.
NOTE: Even though the sample implementation uses Unity, all of the backend logic can be used with any game server and client implementation. For details on how to integrate Amazon GameLift with different game engines, check out the documentation.
There are a few key things to consider when validating if Amazon GameLift is a good fit for your persistent world needs.
As this is only a sample implementation, you always need to do your own performance and load testing and validate edge and error cases.
The solution guidance has been tested to work with the following configuration and results:
Configuration:
Results
The project contains:
Backend
)LinuxServerBuild
)UnityProject
)The architecture diagram introduced here describes all of the components of the solution, as well as step by step data flows.
git clone git@github.com:aws-solutions-library-samples/guidance-for-persistent-world-game-hosting-on-aws.git
npm install -g typescript
to install Typescript, which is used by the CDK application. See Working with CDK In Typescript for more details../downloadAndSetupUnityDependencies.sh
in your terminaldownloadAndSetupUnityDependencies.ps1
in PowershellUnityProject/Assets/Dependencies/GameLiftServerSDK
)UnityProject/Assets/Dependencies/AWSSDK
)UnityProject/Assets/Dependencies/
)UnityProject
)
Before starting, navigate to the repository root in your terminal and open the project in your favourite code editor.
Backend/bin/persistent-world-gamelift.ts
)
deploymentRegion
variable to your selected region for the backend services and Amazon GameLift resourcesBackend/lib/persistent-gamelift-stack.ts
)
fleetLocation1
, fleetLocation2
and fleetLocation3
to your preferred Amazon GameLift -supported Fleet locations where you want to host worlds. NOTE: The first location has to be the same region which you configured in step 1 (Home region).cd Backend
)npm install --force
to install all required packagescdk bootstrap aws://ACCOUNT-NUMBER/REGION
where you replace ACCOUNT-NUMBER and REGION with your AWS Account number and the region you selected in step 1UnityProject/Assets/Scripts/Server/Server.cs
in your favourite editorbackendRegion
to the region you selected in step 1 (in case you changed it)SERVER;UNITY_SERVER
to the symbols. Remember to select Apply after setting these!cdk synth
to create the CloudFormation templatescdk deploy PersistentWorldFleetRoleStack
folder to deploy the Amazon GameLift fleet IAM rolerole_arn
value in LinuxServerBuild/amazon-cloudwatch-agent.json
with the one created by the stack. You will find this as an output of the CDK deployment on the command line (under "Outputs"). This is used later on to configure CloudWatch Agent to send server logs and metrics to CloudWatch.cdk deploy PersistentWorldGameliftStack
to deploy the backend and Amazon GameLift resourcesPersistentWorldGameliftStack-WorldsConfiguration<STACKIDENTIFIER>
table and select "Explore Table Items"sample_world_item.json
which you can modify, and create worlds by running aws dynamodb put-item --table-name PersistentWorldGameliftStack-WorldsConfiguration<STACKIDENTIFIER> --item file://sample_world_item.json
. Make sure to replace the table with your correct table firstOnce you have covered all of the above steps, you should see worlds being provisioned by the backend in the GameLift Fleet to the defined locations in the Amazon GameLift console. NOTE: The default CDK configuration only has 1 game server instance in each location, which allows you to host 2 game worlds in each location. You can configure the fleet to host more worlds on a single instance, as well as adjust your instance scaling as you please. You might need to request a limit increase through AWS support to run more instances.
To test the setup end to end, run two clients that will connect to the same game session.
ApiEndpoint
to the value found in the Outputs of the PersistentWorldGameliftStack in the CloudFormation console, or as an output of the CDK deploymentIdentityPoolId
to the value found in the Outputs of the PersistentWorldGameliftStack in the CloudFormation console, or as an output of the CDK deploymentregionString
to your selected home region (in case you changed it)You can run headless bot clients across different regions to load test your game on AWS Fargate. The UnityBotClient folder contains all the required configuration to set this up. Please refer to the Bot Client Readme) for deployment details.
Many games require dynamic worlds, which scale based on player demand. You could have for example a "Lobby" world in all of the locations of the multi-region fleet, and new instances of the lobby are required as previous ones are filled. You also need to be able to scale down the lobbies where there are no players left.
For this use case, it's not efficient to manage separate lobby worlds as individual configuration. You can instead use the dynamic world configuration by defining the following string value in a world item in the WorldConfigurations table: key: DynamicWorld value: YES
Key things to understand on the dynamic world implementation:
The serverless backend deployed with CDK is responsible for two main functions: provisioning and managing the game worlds and supporting client requests for listing and joining worlds. Serverless backend resources are deployed as part of the Backend/lib/persistent-world-gamelift-stack.ts
stack.
The WorldManager function (Backend/lambda/world_manager.py
) is triggered every 1 minute by Amazon EventBridge. It first queries Amazon GameLift for running sessions and updates this information to the WorldSessions table. It then checks the WorldsConfiguration DynamoDB table for worlds that are not yet running based on the information from the previous step. It then provisions these worlds in the configured locations using the Amazon GameLift APIs.
The game client communicates with Amazon API Gateway by signing requests with Amazon Cognito Identity Pool credentials. There are two request types: ListWorlds and JoinWorld. The game client has a simple UI where the player can select to list worlds in a selected location. The backend function (Backend/lambda/list_worlds.py
) will list the worlds by querying the WorldSessions table. This way we don't need to query the Amazon GameLift APIs directly, and have a more scalable implementation. After listing the worlds, the player can select to join a world. This will trigger the JoinWorld function (Backend/lambda/join_world.py
) which tries to create a player session in the selected world. It will also atomically increase the player count in the WorldSessions table by 1. If successful, player will receive the Player Session ID together with the port and IP of the game server. It connects over TCP to the game server and sends the unique Player Session ID for session validation.
The two CDK scripts defining all the Amazon GameLift resources are Backend/lib/persistent-world-fleet-role-stack.ts
(only includes the Amazon GameLift Fleet IAM to allow configuring it for metrics and logs), and Backend/lib/persistent-world-gamelift-stack.ts
(includes all the other resources).
The main Amazon GameLift resource is the multi-region Fleet, that runs the headless Unity game server build in all the locations defined in the CDK template. It hosts two game server processes per C5.xlarge instance, but you can configure this to your needs to up to 50 processes per instance. We pass the logfile parameters as well as the hosting port information to the instances in the launch configuration. This will allow the processes to run a TCP server on the appropriate port, and allow the CloudWatch agent configuration to send the log files to to CloudWatch Logs.
The Fleet IAM Role is deployed separately, as it needs to be configured for the CloudWatch Agent before uploading the game server build. The role is then used to push CloudWatch logs, process level metrics (through procstat) and any custom metrics you want through the StatsD agent running with the CloudWatch agent. The Unity sample includes a simple StatsD client that sends some sample metrics to CloudWatch.
Key server code files in the Unity Project
Assets/Scripts/Server/Server.cs
: Processes all the client messages and manages the player session. Also uses the Fleet IAM role to check for world termination requests from DynamoDB, as well as to read and write player position data to/from DynamoDB for persistence, using the local copy of the global table.Assets/Scripts/Server/NetworkServer.cs
: Manages the player TCP sockets.Assets/Scripts/Server/GameLift.cs
: Integrates with Amazon GameLift using the Amazon GameLift Server SDK.Assets/Scripts/Server/SimpleStatsDClient.cs
: A simple sample implementation for sending custom metrics over UDP locally to the CloudWatch Agent process that sends the metrics to CloudWatch Metrics.Key client code files in the Unity Project
Assets/Scripts/Client/Client.cs
: Connects to Amazon Cognito for a new identity, or requests credentials for an existing identity stored in PlayerPrefs. Requests backend API calls from the BackendApiClient, requests a connection to the server and manages the player and enemy game objects, and processes all network messages.Assets/Scripts/Client/NetworkClient.cs
: Manages the TCP client connection to the serverAssets/Scripts/BackendApiClient.cs
: Manages the signed API calls to the backend API for listing and joining worldsKey shared code files in the Unity Project
Assets/Scripts/NetworkingShared/MessageClasses.cs
: Defines the SimpleMessage class that is used in the sample to transmit all messages between the client and the server. These messages are serialized and deserialized using simle C# serialization. You might want to look at engine specific netcode options or 3rd party solutions for more optimized package sizes.Assets/Scripts/NetworkingShared/NetworkPlayer.cs
: Manages the spawning, positions and interpolated movement of the player characters both on the server and the client sideAssets/Scripts/NetworkingShared/NetworkProtocol.cs
: Manages the message serializing and deserializing on the server and the clientThe game server processes will do a refresh every 24 hours by terminating themselves after 24 hours runtime. It's a common practice to refresh your MMO game servers on a daily basis. You can do this in a more sophisticated way through a scheduled maintenance that happens every night for example. World Manager will automatically replace the game server processes that were terminated, and the new one will have access to all the same persisted data (in our case, the stored player location).
These rough cost estimations are calculated for 1500 concurrent users, which typically would mean around 150k monthly users. The estimations are done with the pricing in US-East-1 region at the time of calculation, and with the feature set of the sample solution. You will see different total costs in your own implementation based on how your backend, database, and compute resources are used. As all of the components are pay as you go, and scale to your demand, you can see a linear scale up and scale down of the costs based on your concurrent users, with some of the services showing reduced tiered costs for higher usage.
Key data points used:
Cost component | Total monthly usage | Monthly cost breakdown | Monthly cost total |
---|---|---|---|
ListWorlds requests | 6000 API Gateway requests per hour, 6000 Lambda requests at 128MB per hour, 6000 DynamoDB read request per hour (1KB) | $4.38 (API Gateway) + $10.01 (Lambda) + $0.55 (DynamoDB) | $14.94 |
JoinWorld requests | 6000 API Gateway requests per hour, 6000 Lambda requests at 128MB per hour, 6000 DynamoDB write request per hour (1KB) | $4.38 (API Gateway) + $10.01 (Lambda) + $5.47 (DynamoDB) | $19.86 |
WorldManager invocations | 60 Eventbridge events per hour, 60 Lambda requests per hour (512MB, 5s), 60 DynamoDB reads per hour (1kB), 600 DynamoDB writes per hour (1kB) | $0 (EventBridge) + $1.83 (Lambda) + $0.56 (DynamoDB) | $2.39 |
PlayerData database access | 6000 reads per hour, 18000 writes per hour (3x for global table replication) | $0.55 (DynamoDB reads) + $16.43 (DynamoDB reads) | $16.98 |
Amazon GameLift compute and data transfer | 5 C5.xlarge instances, 1500 x 5kB = 7500kB / second | $847.16 (instances) + $1726.55 (data transfer) | $2,573.71 |
CloudWatch Logs and metrics | 100MB per hour (rough estimate), 40 custom metrics | $36.32 (Logs) $12.00 (Metrics) | $48.32 |
DynamoDB storage | Total of 5501MB storage | $1.38 | $1.38 |
Cognito Identity pool | No costs | ||
Total Monthly | $2,694.56 | ||
Total Monthly per CCU | $1.80 |
To clean up all the deployed resources, it's a good idea to first mark all your worlds for termination (TerminateSession: YES
in DynamoDB WorldsConfiguration table). While it's not a required step, you would do something similar in any automation you build for terminating Amazon GameLift fleets, as you want a controlled way to terminate the sessions. Once you have all the worlds terminated, navigate to the Backend
folder and run the followind command to clean up the CDK stacks:
cdk destroy --all
This example is licensed under the MIT-0. See LICENSE file.