Remote control server as an API for custom interfaces

[PeridexisErrant]

This issue is to prompt discussion. A concrete proposal will be drafted following a range of comments, as I expect implementation will be pretty tricky.

There are many programs that do or would like to support remote control of DF - from simple ssh into curses-mode, through dfterm3 and df-anywhere, to df-ios and perhaps someday Armok Vision. To date, these programs have - with the notable exception of DF-iOS - simply passed keypresses along to DF. While this works, it also locks in the present keyboard-driven, nested-menu UI style. I and most regular players like this, but many others are entirely put of DF! So to minimise duplication of effort, I propose that DFHack should develop a server plugin that can translate external calls into actions in the DF interface. This would complement remotefortressreader (which serves DF state), and between them custom interfaces for DF would finally be within reach.

If a few - admittedly difficult - properties can be maintained, this will enable a wide range of new tools and playstyles.

• platform-agnostic: it should be usable from any language, local or remote applications, etc. JSON and sockets seem good.
• stateless: implementation would have to have to go to the main map between command-transactions I think, to avoid all the global state problems. Commands would eg. always refer to map coordinates or unit ID, not cursor coords - the client-side UI might not even have a cursor!
• transactional: if commands are batched into logical chunks and do not depend on state, this largely solves multiplayer DF as well as unreliable connections. When independent actions don't interfere, players can work similtaneously on different parts of the map (cf @warmist's old multiplayer idea).
• error-aware: commands should always return a status, especially on error. Suffice to say that commands should return an error instead of putting DF in an invalid state, or if they are not executable for any reason (eg placing chairs, and none are available; attempting to change labors on a deceased unit; etc.). This does not need to be perfect, but should include sufficient reporting to be useful to client applications for the player interface.
• well-defined: if the total API is well-defined (eg as all allowable JSON in each direction), then middleware can be written to add functions such as user action permissions in multiplayer, retry-on-error handling, and so on. These features are beyond scope of the plugin, but would be enabled by a good design.

Questions or comments would be awesome :smile:

[warmist]

I was thinking about something like this. Though my idea was something along the lines of "transactional core"+ "fast viewer". The reasoning is that most of "viewing" needs to be up to date and can have dropped packets etc. but the input needs to be exact. This ties very well into TCP vs UDP choice.

Similarly here: if we use JSON for everything it will probably hit performance issues. IMHO we should use the protocol buffers or better yet (with new df and dfhack using new visual studio) move to even faster (de)serialization.

That being said as with all things relating to performance it would need to be profiled. But that would only be visible later (when e.g. somebody builds a client that shows history of a selected units clan and dfhack sends MASSIVE JSON as a response). Also i'm not an expert in all json things and imo it's a very bad format good only for html era javascript nonsense :)

As for other parts: it would be awesome. I think it already mirrors a lot of what mifki done with dfremote (https://github.com/mifki/dfremote).

[PeridexisErrant]

Good to have an issue to centralise discussion then!

I think we had roughly the same idea actually - I was thinking that the fast viewer (ie remotefortressreader) is a separate plugin, and this only sends commands. If you want to get any data back (except command status), use a different plugin! And yes, same idea that viewing can be out-of sync or stuttery without serious issues, but commands must be transmitted at high quality.

The advantage of JSON is that everything can speak json. Websites, scripts, native apps, etc... And if it's only sending commands and returning the status of each, I think the total volume would be low enough to be OK. But I haven't written anything like this before, hence the discussion!

This is mostly a subset of Mifki's dfremote functions, with (I think) some extra handling to enable multiplayer control (stateless/transactional/etc). I'd love to see more of his stuff merged into mainline DFHack though, especially TwbT!

[figment]

Just to poke my head where it doesn't belong as I probably wouldn't use this directly and am not too familiar with it.

If I were to make a suggestion then I would very strongly consider one of the common RPC libraries such as thrift or grpc that supports many languages, are reasonably standard and are arguably high performance remoting libraries. Thrift has lua bindings while grpc does not afaik. Now there are pros and cons to this but it would in theory provide a decent standard to work with multiple languages which would be a reasonable for most devs and minimizes work the network layer work.

I've used thrift with C#, python and node.js and it basically worked as needed but its not perfect. node.js was used with react/angular for realtime html dashboards. REST with json isn't a bad but you have more overhead than with a binary protocol in theory. If I were doing a realtime website, I'd consider doing a mid-tier node.js server or something rather than trying to use df directly as the webserver but I suppose it would depend on what was available. This isn't sexy to install though for end users though.

Having said that these libraries are not panaceas and can easily be used in bad ways but could also make it easier to implement. I suppose you can just do a rest api with json but I think i really does depend on what the goal is.

[lethosor]

For what it's worth, I experimented with a remote DF server (like dfterm3) a while back, and found that a binary protocol for screen data worked best. I used JSON for almost everything else (well, I planned to), and routed everything through a server instead of having DF interface with clients directly (which made things simpler on the DF side, and provided better feedback when DF crashed).

JSON does have a couple advantages - it's widely-supported, like PE mentioned, and we already have a JSON library included in DFHack. I doubt that the (size) overhead of JSON over other formats would be that significant compared to screen data. Obviously, massive responses are something to be avoided, but I'm not convinced that anything large enough to cause serious performance issues when serialized to JSON wouldn't also be slow when serialized to other formats.

[PeridexisErrant]

The idea here is that all the high-volume stuff - screen data, item positions, whatever - would be handled by other tools.

For passing around commands only, I think JSON is the best option. The payloads are small - a very high estimate is two packets per second, each a few KB at most (coordinates, action names, context object of some kind...).

And IMO making it as easy as possible to write middleware for this is really important, since that's where client or server features come from, whereas this proposal is basically for a standardised protocol.

[warmist]

So how would it look like? E.g. i want to have a "pause" thingy. Would there be - pause/unpause commands or set/get commands? How would the json look (how much metadata?) Also about the transactional part - do we have json with "Start transaction" or is it just a feature that e.g. build 10 beds at coords [pos1,pos2,...] either queues them up all or fails (and does nothing).

[PeridexisErrant]

The schema isn't finished, but it should have a couple of properties:

1. Idempotent where possible - instead of "toggle pause" or "increase zoom", "set $state to $value".
2. No implicit state, ever. So a command object starts with the mode (main menu, dwarfmode, adventure, legends, worldgen, etc), the the context (eg dwarfmode menu, dwarfmode mainscreen). Includes absolute cursor coordinates for each step where that's required, and so on. This would be a PITA to write by hand, but it's designed for machine validation.

3. Transactional, meaning that each command object (IE json object received) either succeeds or fails entirely. The size of each command is basically up to clients.

   As an example, suppose I'm queuing armor production (and want a full set or nothing). Client A is single-player, and values fast feedback - so each piece is sent as a separate command. Client B is multiplayer, and batches the armor pieces into a single command - meaning that displayed state is speculative or delayed, but avoiding partial results. Fortunately, our server doesn't know or care about the client - just executing a command object, and returning a success or failure diagnostic.

I'd like to build a toy prototype before finalising the schema too, which is likely to take a while.

[RosaryMala]

My initial reaction on seeing this was the thought that protocol buffers work very well with what we have already, but there's merit to the JSON idea.

The problem with JSON, though, is that unless we want to build a webserver into DFHack, you'd need a middle-ware program anyway, in which case, you may as well sick to protobufs

[PeridexisErrant]

I'm closing this issue for now, as there is no immediate demand for the feature and the people implementing remote control for DF seem to be doing fine without it. If in future a standard approach is needed, it can be decided and implemented then.