search

thmsndk / Screeps3D

A 3D client for the MMORTS Screeps.com
89 stars 17 forks source link

Support Replays / RoomHistory #88

Open thmsndk opened 4 years ago

thmsndk commented 4 years ago

It should be possible to support RoomHistory.

There are endpoints available https://screeps.com/a/#!/history/shard3/E19S38?t=12117800 shows history for tick 12117800 and a request is made to https://screeps.com/room-history/shard3/E19S38/12118000.json the response we recieve contains a list of ticks, with the state of all objects in that tick. We recieve a total of 100 ticks in one base payload.

{
  "base": 12118000,
  "room": "E19S38",
  "ticks": [
    "12118000": { "5bbcae129099fc012e63861d": { "energy": 12345, ...}}
     ...
  ] 
}

The user/owner can be found with https://screeps.com/api/user/find?id=UUU where UUU is the guid in the property user for a creep.

For private servers a history mod is required

Perhaps It could detect if history is supported or not and enable / disable based on that?

Ideas and suggestions are welcome. Do we want to support multiroom history? perhaps history from one room and neighbours? Could probably also reconstruct the data to render nuke arcs without the external api calls perhaps?

AlinaNova21 commented 4 years ago

Note that API differs on private server MMO:

GET /room-history/<shard>/<room>/<tick>.json

PS:

GET /room-history/?time=<tick>&room=<room>

Also, tick count depends on the server, see GET /api/version serverData.historyChunkSize for size

thmsndk commented 4 years ago

Ags also mentioned the following on slack

The amount of ticks we recieve are variable, see historyChunkSize in /api/version image

thmsndk commented 4 years ago

I'd like to gather input on how this should work.

If you go to "history" mode where you can move X ticks back and forth in history and change the tickrate is it the "entire" server that goes into "history" mode or only specific rooms, with the other rooms ticking along with live data?

should it be a sphere around the player that is fetched history for?

I've also downloaded some history data from a few rooms for the tigga adventure on shard1. I assume once we can load history from the server in a "runtime" fashion. we also want the possibility to load history from disk, thus allowing the player to function like a replay viewer of past history that you have stored on your harddrive or in the cloud.

Any thoughts, suggestions on this history feature would be much appreciated :)

Robalian commented 4 years ago

I think client should go into history mode and fetch history data only from rooms around player camera. Part history and part live data would just look confusing (official client does this, world map panel in room view shows live data, while there is history playing out in a room).

I wonder if we hit any rate limits with this. If so, then making a system similar to how videos are buffered should be considered. So some amount of history would be buffered and played. If there is not enough data yet then it's paused, perhaps with some youtube style buffering animation. And you could manually adjust "resolution" - amount of rooms history is fetched for - based on how fast history is being replayed.

thmsndk commented 4 years ago

Yeah I think we need 3 distinctions, history mode, replay mode and live. The code would be refactored in such a way that "something" can feed data into the renderer, such that it is decoupled.

History mode

This mode is "room history" like the official client. An area around the player location fetches history .json files "live" from the server. depending on what tick we are currently on.

Depending on how many rooms we need data for and what tickrate we have set for the playback, we need to prefetch, or "buffer" theese files. We might need to cache theese files on disk as a temp thing or hold them in memory. depending on how large a buffer we need.

I definitely like the idea of a buffering video / stream system, so we can fetch them async when the tick gets close to the end or if the player clicks on the "select tick" progress bar repeatedly.

Replay mode

Replay mode is like history mode, but here a user can point to a directory or zip with replay data already downloaded. depending on how old the replay is, we could supplement with data around the player. or just render the rooms in a "No Data" state

The big hurdle with this issue is rewriting everything allowing us to feed data into the drivers/renders as if it was coming from the websocket.

I totally agree that mixing live and history makes it confusing, and we should probably render a "No Data" overlay on rooms that have no data, or somehow indicate in another way that we are in "history/replay" mode :)

thmsndk commented 4 years ago

In regards to rate limits #218 would probably be the solution, disabling rate limits if they are hit or getting close to be hit. if for example we are hitting rate-limits and thus the buffering becomes really slow. I never noticed that I hit any rate-limits when I downloaded the history from the tigga shard1 colonization attempt though

thmsndk commented 4 years ago

In broad strokes, what needs to be done with our current implementations

We need a new component able to download/load json history files either from disk or live requests, this component should be "buffer" aware such it can request new increments of data depending on chunck size.

We need a new component able to read/parse json from history files and direct the data to the appropriate sub components.

We need to be able to feed data into the following components

The following components might need to be disabled during replay