Dynamic Realm Support

[pramukta]

This is a great looking project. It's really great to see a nice minimalist WAMP router as a static library. I am somewhat in the same boat as @estan who opened #9 but also could really use automatically managed realms as well. So by that I mean that a realm would be created when a client tried to join it and would be thrown out when the last client left it. I think this would be really convenient for people doing WAMP development as well. What would you guys think about incorporating something like that into the codebase?

My C++ is rusty almost to the point of being non-existent but I did manage to put together a proof of concept branch. It definitely would require someone more knowledgeable to go over but the general approach I tried was:

1. Allow src/bonefish/router/wamp_routers to track an io_service reference and use it to create and add routers/realms in get_realm
2. Expose a has_sessions() method on the src/bonefish/router/wamp_router
3. Have the on_close and on_fail handlers from websocket_server remove the realm if has_sessions() returns false.

I modified examples/integration.cpp to run this "dynamic" mode. The standard bonefish binary works exactly as before. I don't think this is PR worthy yet but wanted to see what folks thought. And definitely point out anything i'm thinking about wrong. Thanks.

[jpetso]

Just out of interest, what kind of use cases does this feature enable? Does Crossbar have anything comparable?

[pramukta]

The dynamic realm support for me means that I can create an app as a collection of WAMP components, put the server side components as a docker container(s) or similar, and have users spin up their own copie(s) by running the container, and pointing it to a new realm. But I think it becomes useful anytime a user is managing their own collections of components. Or for just lazy development.

The router that used to be included in AutobahnPython had the ability to dynamically create realms. Crossbar does not have this feature (although I believe the code is in there just disabled). Instead they used to have an unsupported management API which they have changed / removed documentation for. I believe they are moving to a model where they host a management interface that your crossbar instance connects to and then allows you to create and configure realms from there.

[davidchappelle]

I am not sure that dynamic realms are something that we want to support at the present time. A realm is something that is usually statically provisioned/defined so that it can be administered and even potentially utilized by multiple routers. When you define a realm you define other characteristics associated with that realm. For example, the roles supported.

Furthermore, when you define a router, you need to also define the realm(s) that it is associated with. You also need to define a number of other characteristics regarding transports, authentication, etc. Some of these configuration parameters are not yet in place but will be added in the future. There are of course other concerns here as well like DOS attacks that can occur by allowing arbitrary hosts to dynamically allocate routers/realms.

I think that if we were to support dynamic realms we would need the following items in place first:

1. A proper configuration mechanism for the bonefish daemon as discussed in #13.
2. A proper administration interface that can be used to authenticate a client and allow it to perform configuration. This would allow clients to dynamically allocate routers/realms.

In the meantime, a better suited workaround for what you are trying to achieve would be to simply spin up new instances of bonefish. This would allow you the ability to set the realm/port of each bonefish instance and give you the dynamic behaviour that you are looking for. Since the clients would need to know which realm to connect to you must be preserving/passing that information anyways. So adding the port along with the realm name shouldn't pose a problem.

[davidchappelle]

In fact, doesn't each docker container have its own ip address? If so, then each docker instance can just run its own bonefish router on its own realm. In fact, the realm need not even be unique in this case either.

[pramukta]

That's a fair position, and the one that I think the crossbar folks are taking. I can spin up one router per container, have docker map ports, and track those. I actually had been using the exact approach you have suggested using crossbar.io before but found that the memory footprint of many instances was quite wasteful for the amount of work they were doing. Maybe bonefish would be better on that front.

It's a bit unfortunate since the core WAMP protocol elements are so elegant and convenient and most security can be handled at a proxy layer for untrusted clients (ie the browser most of the time), avoiding having to build out extravagant protocol and server features.

I wonder if as a compromise you'd be willing to allow the dynamic realm support as a library feature but not a bonefish server feature? In the current proof-of-concept, daemon/bonefish behavior is unmodified. It's only the integration example that does any dynamic work.

It's also cool that bonefish lets you use it as a library so easily. If you intend to keep that philosophy, I was starting to look into how to create python / nodejs bindings. This could be an interesting direction that would help out folks that don't need crossbar.io's rich feature set without putting pressure on bonefish to remain very simple.

[pramukta]

Docker containers don't typically have their own IP addresses. They do map ports between themselves and the host though.

[jpetso]

I'm thinking maybe the library can offer optional callbacks to notify when certain events are happening, such as "client opening session", "client joined realm" and "client left realm", with the number of current clients being passed as callback argument. The library user would then have the option to then invoke standard router/realm management calls instead of relying on automatic magic.

One would have to check to make sure that asynchrony is not a problem for the logging-in part where you wanted to create the realm in between being contacted by the client and bonefish checking whether they can join.

I think python and/or node bindings would be great. The one thing I would want to pay attention to is integration into Travis CI, since many users (including us at Topology) will not have a use for that directly so we should make sure it works via automated builds.

[davidchappelle]

The executable and memory footprint of bonefish is very small:

Executable

dchappelle@orion:build$ pwd
/home/dchappelle/projects/bonefish/build
dchappelle@orion:build$ cmake -DCMAKE_CXX_FLAGS=-O3 ../
...
dchappelle@orion:build$ make
...
dchappelle@orion:build$ du -m daemon/bonefish 
3   daemon/bonefish

Memory

dchappelle@orion:types$ ps -o rss,sz,vsz $(pgrep bonefish)
  RSS    SZ    VSZ
 2372  5090  20360

Likely significantly less resource consumption than crossbar.io.

My main concern with allowing the dynamic realm support at the library level, is that it may prevent obstacles for future work if we have to continue supporting it. Once it is there it may be hard to remove at a later point in time if other projects have a dependency on it. I think that based on the small footprint of the daemon, it may be a better option to just deploy multiple daemons to address your use case for the time being. Our plan is to definitely continue to allow library level access and is in fact our primary use case. This is one of the reasons that no progress has yet been made on providing a proper configuration framework for the daemon. I think that providing bindings may be useful at some point in the future for embedding a bonefish instance inside another product written in a different language. If you do decide to go down that road, please open an issue first and initiate that discussion. I believe we already have a really nice framework for creating bindings that may prove to be useful.

[pramukta]

Ah yeah, the optional callbacks approach would be really awesome. I'll try and look into it some and see if your concerns can be addressed though, as I may have mentioned before, my last foray into any C++ was >10 years ago so it's slow goings. I apologize in advance if at first it looks like I'm writing C and pretending it is the same thing.

I'd also be interested in trying to help out with #9 since that is actually a bigger obstacle for us using bonefish than this ticket. It sounded like there was some work done on that? It would be awesome if even an incomplete branch could be pushed. But will move that comment to the appropriate place.

That's awesome about the resource consumption, and great to hear that the lib is your primary use case. I would be very interest in the bindings, I may be overloading myself a bit but will open a ticket for that probably later today unless someone beats me to it.

Thanks for the quick responses with everything, really appreciated.

[davidchappelle]

@pramukta We appreciate your interest in the project. I am going to push a patch later tonight for #9. If you are willing to pick it up, test it, and help with any bug fixing that would be greatly appreciated. I'll post here once it is pushed.

[pramukta]

I think the conclusion was that this wasn't such a great idea. So closing.