Discord

Rust ro 3 years journey blog post

Welcome to rust-ro!

rust-ro is a Ragnarok MMO Server implementation written in Rust.

This project does not aim to compete with herculesWS or rathena.

Although the architecture and technical decision of this project are very different than rathena or hercules, both projects are still a source of inspiration and remain source of truth for game behavior.

In addition, this project kept some concept of existing implementations: for example this project support same scripting language for NPC meaning that existing scripts should work on this implementation and use the same database structure than rathena and some naming convention were kept.

Architecture and technical decision are documented here

1. Ambition

The project started on August 2021 with the ambition to being able to connect on the server and navigate across maps to see monsters.

Today February 2023 a lot of features have been added, at the end of readme. My ultimate goal would be to have a fully playable implementation for PRE-RE, supporting packet version 20120307.

I am working on this project for fun and also to provide a more accessible way than existing implementation to understand how Ragnarok game works for educational purpose. Each feature is covered with tests to provide internal documentation.

2. About Packet Version

The packets parser, builder and serializer are all taking packet version as parameter. The packet db also support condition around packet attributes and packet id

Although I mentioned above wanting to fully support packet version 20120307, this implementation can support any packet version, it is just I am testing exclusively with a robrowser client using this packet version and thus i am implementing only packet in this version.

3. Currently Working

Below issues are under active development

• Skills issue - Offensive skills progress
• Status and bonus issue - Stats for each job level progress Stats for each items Stats for each cards

4. Implementation details

To understand what is going on at this project, check the architectures notes.

4.1 Project files structure

• lib: contains either, proc-macro, reusable structure, generated code
  • lib/packets: Structure for packets generated by tools/packets from tools/packets_db file
  • lib/models: Plain old data structure to be reused in multiple modules
  • lib/configuration: Structure for configuration with serde deserializer implementation
  • lib/skills: Generated structures for skills from configuration and also manually implemented skills methods
• server: server core
  • server/proxy: A proxy implementation to be use between a client and an emulator (rathena or hercules) in order to capture packets
  • server/repository: data access layer of the server, any access to dabase is written from this layer
  • server/server: global event loop, map instance event loop, persistence event loop
  • server/server/boot: any method require in order to boostrap server state (script loader, maploader, mob spawn...)
  • server/server/model: Plain old non-reusable data structure (queue, cells, etc..)
  • server/server/request_handler: controller layer, any packet receive from client pass to a controller first
  • server/server/script: interface between server and script virtual machine, provides hook implementation of the virtual machine
  • server/server/service: any logic of the game is implemented in this layer
  • server/server/state: Data structure containing game state (character session, character state, mob state)
  • server/util: Any utility methods
• tools: code generator
  • tools/map-cache: generate map cache from map files
  • tools/packets: generate packets structure from database file
  • tools/skills: generate skills structure from configuration file
  • tools/stat-calc: A javascript/html stat calculator, cleaned up to better understand stats and combat calculation. Also added feature to generate test fixtures, to validate our implementation

5. Setup

Here's a list of pre-requisites to run rust-ro:

• Docker OR PostgreSQL 12+ directly on your machine
• Rust - nighly build

5.1 Config

First, make a copy from config.template.json to config.json:

cd rust-ro
cp config.template.json config.json

Inside this JSON, you will find database related variables, game related variables (exp_rate, drop_rate etc) as well.

5.2 Database

The entire database structure was based on rAthena but instead of using MySQL, we decided to go with PostgreSQL. There's minor modifications so far but until we mapped some constraints.

The choice to use Postgresql instead of MySQL was mainly motivated because I know better how to operate Postgresql than MySQL, and know better how Postgresql (mvcc) works internally. In addition past years Postgresql gained more traction than MySQL and its open source model

5.2.1 Setup Database - Using Docker

If you already have Docker installed in your machine, we prepared a docker-compose.yml with all configs ready for your ragnarok server.

Go to /docker and run:

docker-compose up -d

The first time, along with postgresql initdb is run, our custom script init.sh will be execute, it will create ragnarok database and create ragnarok user using postgres user. Then it will create ragnarok table using ragnarok user.

It comes with a default player account with following credentials: admin/qwertz

5.2.1 Setup Database - From binary

If you have PostgreSQL installed in your machine, you will need to log-in into PSQL and create the user, dabase and give the necessary privilege for it:

sudo -u postgres psql

Run the queries below:

CREATE USER ragnarok WITH PASSWORD 'ragnarok';
CREATE DATABASE ragnarok;
GRANT ALL PRIVILEGES ON DATABASE ragnarok TO ragnarok;
ALTER DATABASE ragnarok OWNER TO ragnarok;

After that, exit pgsql and import our /rust-ro/db/pg.sql via cli with:

 sudo -u postgres psql -U ragnarok ragnarok < db/pg.sql

5.3 Running the Server

After we have everyting set-up (binaries and database), we should run server binary to turn on rust-ro.

To run the server binary, you will need a ENV variable called DATABASE_PASSWORD together with your command:

DATABASE_PASSWORD=ragnarok cargo run --package server --bin server

If everything goes right, you should receive something like this output:

2024-02-11 13:45:53.695168 +01:00 [main] [INFO]: Compiled 0 item scripts compiled, skipped 2492 item scripts compilation (already compiled) in 73ms
2024-02-11 13:45:54.976721 +01:00 [main] [INFO]: load 39 scripts in 1104ms
2024-02-11 13:45:55.110070 +01:00 [tokio-runtime-worker] [WARN]: Not able to load boot script: pre-re/warps/other/sign.txt, due to No such file or directory (os error 2)
2024-02-11 13:45:55.113409 +01:00 [main] [INFO]: load 2782 warps in 6ms
2024-02-11 13:45:55.134622 +01:00 [<unnamed>] [INFO]: load 3392 mob spawns in 19ms
2024-02-11 13:45:55.152271 +01:00 [main] [INFO]: Loaded 897 map-cache in 44ms
2024-02-11 13:45:55.378476 +01:00 [main] [INFO]: Executed and cached 1601 item scripts, skipped 891 item scripts (requiring runtime data) in 226ms
2024-02-11 13:45:55.388987 +01:00 [<unnamed>] [INFO]: Start proxy for map proxy, 6124:5121
2024-02-11 13:45:55.389135 +01:00 [<unnamed>] [INFO]: Start proxy for Char proxy, 6123:6121
2024-02-11 13:45:55.389212 +01:00 [main] [WARN]: Visual debugger has been enable in configuration, but feature has not been compiled. Please consider enabling "visual-debugger" feature.
2024-02-11 13:45:55.389241 +01:00 [main] [INFO]: Server started in 2347ms
2024-02-11 13:45:55.389292 +01:00 [main] [INFO]: Server listen on 0.0.0.0:6901

5.4 [Dev] Running tools

So far, we have a few executables being compiled together with the project:

• maps-tool: Generate mapcache files from client data: cargo run --package tools --bin maps-tool
• packets-tool: Generate packet struct from packetdb: cargo run --package tools --bin packets-tool
• skills-struct-generator: Generate skills struct from skills configuration files: cargo run --package tools --bin skills-struct-generator

5.5 Running the Game

The goal of the project is to run all packages from packetver 20120307, we decided to use roBrowserLegacy.

If you have interest to contribute in a client with packetver 20120307, open a new issue and let's make it happen!

5.6 Running the Stat calculator/test case editor

The stat calculator is a highly customized* fork of from https://web.archive.org/web/20090319055622/http://rode.doddlercon.com/db/calc/index.php

*Customized :

• Code has been de-obfuscated to make it comprehensive
• Code has been reorganize so it at can work on both browser and node environment
• Customize calculation behavior in order to get sub calculation output (e.g: get just the weapon attack before applying some modifier)
• Build a test case editor to generate and edit fixtures for unit and integration tests

The idea behind reusing stat calculator is to validate implementation of battle and status calculation.

Output of stat calculator may be false and in this case we will check result against hercules or rathena.

cd tools/stat-calc
npm run dev

6. Developer Notes

• All packets for account 2000000 are handle by this project.
• All packets for any other account are proxied (and display in console) to hercules or rathena.
• clientinfo.xml to be changed to target port 6901

In proxy mode:

• login, char, map server to be running using default ports (6900, 6121, 6122)

7. Progress Showcase (Compilation)

A compilation of progress made so far, click on streamable video below

7.1 Integration of the VM (showing instance and class(npc) variable)

https://user-images.githubusercontent.com/1909074/178155321-d3eeb4b8-32ed-4901-bbfe-b101b1a5a56d.mp4

7.2 Visual debugger

Debug server state with a UI

7.3 Warps

7.4 Mobs

7.5 Proxied packets

8. What has been done? ✔️

Some list of features that was developed so far:

8.1 Tools

• packet structure generator from packet db
• packet parser generator
• map cache generator

  8.2 Server

• proxy login, char and map request to hercules/rathena login, char and map servers
• packet debug
• login
• char server features(create char, delete char, join game)
• move character in a loaded map, synchronized with client side movement (no lag, or teleportation, movement is smooth)
• character position calculation (implementation of client side path finding)
• debug log in in-game chat
• parse scripts (only warps and mobs at the moment)
• warp (change map, reset states)
• display scripts client side (only warps and mobs at the moment)
• visual debugger
• map instances (map are lazily loaded, an instance is created when a player join an non initialized map)
• mob spawn
• atcommand: @go, @warp
• mob move
• NPC scripts (partially: see https://github.com/nmeylan/rust-ro/issues/3) via rathena script lang interpreter
• basis for inventory management
• basis for skills
• basis for consumable item usage
• basis for player attacking mob
• mob drops item on death

9. Contribution

This project is currently half-open* to contribution. The reason is that all basis have not been put in place already and there are many thing to design yet.

However if you are motivated and want to contribute you can take a look to the contribution guide

* contribution can be made under certain condition