WebSockets support from within the server

[rtubio]

Remove the dependency on the pusher.com external service for real-time communications and relay instead on gevent-socketio.

[rtubio]

Currently, the better option for achieving a complete integration of all the services will be the usage of crossbar.io with AutoBahn and WAMP.

[crespum]

Due to the early state of WAMP, the possibility of using ZeroMQ should also be checked. It provides a lot of flexibility and some important players are using it.

[rtubio]

ZeroMQ is a good choice if we can find a library for the browser for JavaScript. The only one I could found was:

http://stackoverflow.com/questions/8145060/zeromq-in-javascript-client

... but, as you can see, it is not very well developed. This is the drawback that I find, apart from the fact that ZeroMQ looks like pretty low level. Could we try to look to set up the environment for ZeroMQ? Just a working example...

I have also found the possibility of using gevent-websocket, with gunicorn:

https://pypi.python.org/pypi/gevent-websocket/

Apartently, it is mature enough and works well, although the documentation is not as good as for other projects.

I am currently setting up crossbar+autobahn and, due to the posts that I am reading, it looks like it works but the documentation is not as good as it could. It is really high level and there is working binding libraries for AngularJS and Python. We can talk about this on Monday next week.

[rtubio]

@xcrespo : we might have some more luck with this project: SwampDragon (http://swampdragon.net/ , https://github.com/jonashagstedt/swampdragon)

1) It permits websocket pub/sub approach, 2) Natively integrated with Django and its ORM, 3) Installation and configuration is way easier,

Drawbacks:

1) It requires running Redis, which looks like a type of database that runs on memory and provides an access to the data based on a key/value pattern. It looks like it uses a direct connection to this server in order to "do the realtime job", 2) Redis limits to the size of the RAM memory of the server the biggest data set available, 3) No direct RPC method callable, the remote update of the database model should be done through POST... (too slow?)

Please, take a look at this new option as well because it just made my day... :)

[rtubio]

@xcrespo (@ajvazquez FYI) : in my opinion, we should keep using the same architecture we have and forget about the native server support for websockets for release 2. The reason is that we should be facing a complete re-factoring of the server code which will add a lot of extra work for which I do not think that we have the necessary amount of resources available.

Therefore, I propose you to continue developing this architecture and include the JRPC internal interface for the de-coupled communication of the protocol with the server. Python 3 is more than necessary in the server since Django 1.7 already implements its own transparent migration scheme and that is an advantage that we must not under-estimate.

Please let me know what you think.

[crespum]

@rtp-calpoly I agree with you. We should keep the server architecture at least for this release.

I've been checking what queries does the protocol make to the web services and at this moment we'd need the following ones:

• StartRemote: everything related to the slot status. It first check if the slot is operational. This could be a method that returns either an error or the following parameters of the slot:
  • slot[0].groundstation_channel.groundstation_set.all()[0].user.username
  • slot[0].spacecraft_channel.spacecraft_set.all()[0].user.username
  • slot[0].state
  • slot[0].end
  • slot[0].groundstation_channel
  • slot[0].spacecraft_channel
• SendMsg: store messages in the database and flag them as forwarded when needed (this last step is not yet implemented)
• Login: validate a user/password combination

[rtubio]

@xcrespo : we will postpone this improvement in the software for future releases.

[crespum]

There are different possibilities when implementing this methods, the protocol repository will implement those detailed below but we are open to other suggestions:

• get_slot_info(slot_id) shall return an error if the slot is not yet operational. Otherwise it shall return the following JSON object:

  {
    "state" : slot[0].state,
    "gs_username" : slot[0].groundstation_channel.groundstation_set.all()[0].user.username,
    "sc_username" : slot[0].spacecraft_channel.spacecraft_set.all()[0].user.username,
    "end" : slot[0].end
  }

• store_message(slot_id, upwards, forwarded, timestamp, message) shall return a JSON object containing a boolean indicating if the message was or not stored {"result" : result}

For logging in we will use the method given by rpc4django: system.login(username, password).