Write or find a python library to gather all parameters which should end up in `mlab-ns`

[nathan-at-least]

This is a subset of #4 which excludes the component that talks to mlab-ns. This just gathers all the relevant details and returns them.

This comment is a good starting point: https://github.com/m-lab-tools/ooni-support/issues/4#issuecomment-43109985

[defuse]

[defuse]

I've created the bouncer-plumbing branch, and in that branch there is a bouncer-plumbing directory which contains two scripts:

• collector-to-mlab: Get data out of an Ooni slice and pass it to Nagios or our fake mlab-ns (currently stdout).
• mlab-to-bouncer: Take data from stdin (or our fake mlab-ns) and generate the bouncer config file.

[defuse]

So far this adds one new dependency. To run the script I had to install:

• sudo yum install python-yaml

[defuse]

So far the collector-to-mlab script gets the correct IPv4 address and loads the oonib.conf and parses it as YAML. I'm struggling to get the .onion, since the collector doesn't seem to be running on our install on mlab1.nuq0t.

[defuse]

For now I'll just test against mlab2.nuq0t, since it is running a collector, being very careful not to break anything. I created #52 to figure out why it's not starting on ours.

[defuse]

Now when you run it on mlab2.nuq0t, it prints that part of the bouncer config file:

$ python getconfig.py 
ihiderha53f36lsd.onion:
   test-helpers: {http-return-json-headers: 'http://64.9.225.221:80', tcp-echo: '64.9.225.221:81'}

Note that I hard-coded http-return-json-headers and tcp-echo into the script. We still have to make it find out which test helpers are running (and supported for collection) and return just those.

[defuse]

Remaining tasks:

• [x] Document that python-yaml needs to be installed to use this script. (moved to https://github.com/m-lab-tools/ooni-support/issues/50#issuecomment-50080420)
• [x] Find out how to tell which test helpers are running and include exactly those in the output. (moved to #55)
• [x] Fix the code's error handling.
• [x] Make it push into our mlab-ns simulator.

[nathan-at-least]

We should check (later) into using the same yaml dependency which oonib uses. It may be stuffed into a virtualenv or other unusual location.

If this approach is working, that's fine, but after we reach a "working deployment" state, we probably want to do a simplification pass on the packaging.

The mlab-ns-simulator installation in prepare.sh attempts to use the same virtualenv as oonib, so we could do something similar to that.

[defuse]

I fixed the error handling, made it return non-zero if something goes wrong, and now it can take the path to oonib.conf on the command line.

[nathan-at-least]

Let's get the scripts which are in the bouncer-plumbing branch into the mlab-ns-simulator branch of our ooni-support repository, then we can close this.

[nathan-at-least]

This is on mlab-ns-simulator.