Create a common test infrastructure

[bzz]

The idea: have a common executable spec for the service which can be re-used\shared to verify high-level correctness of the implementation in any technology stack.

Example: NYTimes Lab has Go software (REST API server, talking to Elasticsearch as backend) with spec in Ruby https://github.com/nytlabs/hive/blob/master/spec/hive_spec.rb

The only requirements are:

• specs to be as close to English\readable as possible
• have minimum pre-requests\assumptions about the environment to run then (ideal - single command, no need to install anything)

@felizbear do you have any ideas how to do that?

[heruka-urgyen]

We need to install a testing framework anyway. I guess we can host it somewhere and run tests remotely.

[bzz]

No, what I mean is - we do not need to do it manually, neither make users do it. It can be perfectly automated and taken care by dependency management, be it NPM, Ivy, Maven or whatever we chose.

Best user experience in the software engineering is a "single command build", no need to developer to do anything, except run the thing.

I'm interested in to perusing this kind of perfection.

In the example from NTYLabs: they assume Ruby+Rspec installed on dev env. Although os x has one by default but I would say it's quite heavy assumption, considering "RVM vs RBEnv".

So options I see here:

• Node + some Rspec clone (any BDD?) - but it very heavy too
• JVM + gradle + ScalaTest (any BDD clone) - it's much lighter, considering that only jvm is pre-reuqest and there is no need for compiler toolchain to install it\its dependecies
• Ruby + Rspec are fine too
• Python?

[bzz]

A lot of fun with the styles of acceptance tests in BDD http://scalatest.org/user_guide/selecting_a_style Please, check it out and share other examples you know!

So I guess by 'spec' I meant an acceptance tests that could be run with on a local machine with the single command like ./test http://localhost:8080 and have least possible number or dependencies, that are required to be manually installed on clean env (by new developer)

[heruka-urgyen]

I assume, Python comes preinstalled not only on OSX, but also on some Linux distributions, so probably it's the least heavy version? That said, I know nothing about Python dependency management and build automation. Can it install dependencies locally (e.g. like node does in node_modules)? Asking for installing JVM is already a lot, so I think it's roughly equivalent to the Node case.

[bzz]

Good point.

The only issue with pre-installed python package in Linux is - version :( it is still mostly Python 2.6 witch is horrible and the least version we should be using now, if not 3 then Python 2.7 (i.e CentOS 6.5, second most popular server linux distro, after ubuntu, STILL ships with python 2.6 by default)

In python there is great tool for dependency management called pip but by default it manages global, system wide packets :( So de-facto standard to solve this (separate, right?) problem is another tool small tool called virtualenv which actually inspired npm behavior I suppose.

Usual workflow with python software looks like:

• user have Python, pip and virtualenv installed
• checkout
• virtualenv <name-of-the-dir> && source <name-of-the-dir>/bin/activate - like creating and activating local node_modules
• pip -f install requirments.txt - think npm install for packages.json

[bzz]

This is how typical very-well implemented dependency management looks like in C++ world https://github.com/google/certificate-transparency#dependencies

[heruka-urgyen]

I can try to make one in node with jasmine, for example. And leave the decision about Python version to you, because I have no clue :)