Viability Problems of Current Stack

[kipraske]

It is has been quite a while since I have revisited this project and I suspect that this is an indication that the current setup is probably not viable. I am opening this issue to discuss what I feel are the major problems and if anyone is out there the best way forward. It is a good time for a reflection.

Problem 1: The tools used for the project were chosen for my own personal enrichment. In hindsight some of the tools don't make too much sense here. I wanted to learn node.js so I used node.js. I wanted to learn backbone.js so I used that too. Same with mongo.db (which is completely inapproprate for this project if you think about it). Unless there is a lot of Brogue-loving web-developers out there I am not going to get much traction on help on this project. The project is not viable without help because of the complete lack of free time I have for the next year or so.

_Problem 2: _ I tried to do too much at once. I wanted to make a DCSS-like server with all of the bells and whistles. Well, that was built on an existing architecture of playing crawl through ssh and using the ttyrecs to record them. The code to change the console information into tiles information could be easily hijacked into javascript and viola you can play just the same as you always did with a pretty web interface. Brogue does not have any of this history at all. In fact, with my very limited C knowledge we hacked together the very core of the Brogue I/O so we could stream this data out in a way we could use. I don't think that modifying Brogue Core I/O is particularly viable long-term.

Problem 3: I wanted to play on the mobile so I made some strange design decisions. Going along with doing too many things at once, choosing to use a DOM console was driven by my desire to play brogue on mobile. I knew this was going to be slow, but I didn't realize how bad it could get with non-multithreaded browsers trying to keep up with all the js logic and DOM repaints. I am looking at you firefox. So what we have is a very impressive DOM system that is practically useless unless you are using a desktop.

_Problem 4: _ It got really complicated. Node is great but man, when you end up with a bunch of things listening to a bunch of other things you can end up with a big old mess like my backed became. It is not as bad as it could be, but the javascript doesn't lend itself to "jumping in"

_Problem 5: _ It won't scale. Not that there ever would be a huge number of brogue fans, but the single-threaded node process would get bogged down under a heavy load. Node may still be a good solution in the end but not for sending MB of data out to the client for everyone.

So with that, I have abandoned this project thinking that we should do something else entirely to solve one thing at a time well. Here are the problems we are trying to solve with a brogue web-server:

We want:

1. Something where folks can play tournaments online and compete
2. Something where you can watch games online to learn from other players
3. Something preferably that works well on mobile
4. Something where it is not very easy to cheat

[kipraske]

So it seems that the thing to get nailed down first is getting the client to work properly on mobile. Using the DOM works, but just isn't going to cut it. Ideally we can use upcoming technologies like webGL to render our console more efficiently, but we won't want to build that ourselves...

The most reasonable approach would be to compile the game with Emscripten with those SDL addins and have Emscripten figure out how to deal with output, I/O, and game-processes on that end. The big problem with Emscripten is you need a single game loop so the Emscripten engine can properly create a non-blocking javascript loop.

Some research I have done into this makes this seem promising: https://github.com/kripken/emscripten/wiki/Asyncify where you replace all unix sleep-like calls with an async version emscripten_sleep. Brogue utilizes SDL_SLEEP in LIBTCOD to throttle the I/O loop so this should work.

In addition once that is in place one could start building a server by just sending updates to the broguesave and broguerec files. Doesn't really prevent cheating, but what is preventing cheating right now?

And my initial attempts fail pretty spectacularly. The main issue being to go the Emscripten route we need to also port LibTcod to emscripten which should be possible but very hard for someone who is not familiar with any of this stuff.

[kipraske]

A final addendum. One of the reasons Crawl is able to handle I/O so much better is that very little data needs to be passed to the frontend to render the tiles. You only see the 20x20 grid at any time, and things like your health and status are all sent separately. In this WebBrogue, the healthmeter, the inventory etc are all just text representations which don't allow us to offload any of the data being sent to be cached on the front-end.

The major failure of this project was simply the jaw-dropping amount of data that gets sent from the server to render the whole scene, and I am not sure how to get around that without simply having brogue run in the client instead.

[bleakley]

Hi kipraske,

Doesn't it prevent cheating? All non-cosmetic RNG is tied to the seed, right? Replays currently work by re-inputing the same keystrokes that the player originally took, in the same seed, resulting in the same gameplay.

So if brogue ran in the client I could request a seed number from the server and then play brogue normally (gaining the benefit of increased performance, flickering lights, water animations, etc), while sending each one of my keystrokes to the server updating my broguerec. Each observer would also receive my keystrokes, updating the real-time recording they are viewing in their own client. If I tried to cheat I would receive an out-of sync error.

Granted, even if the seed wasn't viewable during a tournament, I could just CTRL-SHIFT-I and read the seed the server sent my browser. I could then boot up the desktop version of brogue in debug mode and wormhole down to d26 and look at all the items. Is that the kind of cheating you are worried about?

That could never really be prevented, even if the seed was hidden and the entire game ran on the server, because a cheater could just enter the same tournament multiple times with fake accounts.

The path you outlined in your second post sounds like the best solution, short of porting huge amounts of brogue UI code to javascript (which would preclude a simple update of web-brogue once 1.7.4 is updated).

I am not an expert in Emscripten/SDL/etc, but I would be willing to assist with this (ambitious) project. I think web-brogue has proven itself to be popular, and something worth building and maintaining.

Thanks for all the work you have done so far! I've really enjoyed web-brogue.

[flend]

Hi all.

I wouldn't paint such a desperate picture :) Let me defend the architecture of your project to you :)

The current live version of brogue, hosted out of the UK, runs on a $5 / month DigitalOcean instance just fine. It has resources for about 5-8 concurrent games of brogue, assuming all are on autoexplore (worst case). The limiting factor for scaling appears to be the CPU use of brogue itself, rather than anything to do with the server architecture. Bandwidth consumption is the next largest scaling factor, but again a $5 instance (1 TB/month) will support about 10 games permanently on autoexplore and there are options to reduce this (e.g. like crawl render only the start and end frames when autoexploring, with footstep graphics to show the route taken).

To date there has not been any reason to go beyond a $5/month single CPU instance since the usage has never maxxed out the server (peak is typically 3-4 games in progress simultaneously). An 8-core CPU instance could go up to 40-50 games on a single server and still fall within the bandwidth cap at that grade. That would be practical but expensive on DigitalOcean but again, this is a brogue problem, not a web-brogue problem.

In the case of needing to scale beyond a single server, I would go for a multi-tiered approach with a series of front-end servers handling the web-sockets, with a series of back-end servers running the brogue exes. I think this could scale horizontally and roughly linearly. I have a presentation on YouTube about this: https://www.youtube.com/watch?v=fvTAIIsnBmA

In terms of the current architecture which effectively sends the screen updates from brogue to a thin web client, rather than sending state updates (monster moves here) to a thick web client, I think it's pragmatic. If we were writing a commercial game that we expected 1000s of people to play we wouldn't do it this way, but it works well and scales to likely usage fine. The huge advantage is that it requires comparatively little effort to write the web server and client and that updates to brogue can be handled with almost no update to the web server. The disadvantage is primarily bandwidth usage, as above, but again I think practical usage levels will not hit hosting bandwidth caps, and bandwidth will get cheaper faster than the brogue community grows :)

In terms of choice of technologies, I think node.js is just fine. Again, if I had to write a server for 100s of connections, I would use Java since the JVM is better understood, at least by me. But people use node for this sort of thing and the server has performed fine so far. The database requirements of web-brogue are so light that any old database is fine and mongodb is well supported by node.js.

The choice of the DOM model for the web client is a bit more controversial and it does practically limit the client to use on Chrome, other browsers being too slow. Although many people are happy with it at the moment, I am considering rewriting the front-end to use HTML5 console / webGL, probably via pixi.js. Having just completed a HTML5 console 7DRL, I don't think this would be particularly difficult. This is the only large piece of work left on my todo list.

As I see it, the UK web-brogue server has proved to work and be popular. I am pretty confident that a single server in each region using the current architecture will satisfy the world's current demand for web-brogueing :)

[kipraske]

Thanks flend & bleakly. Pretty much everything you have done for this project has made it much better. I am going to go adjust the issues now so we are closer to the reality of the project now.