Develop API Testing Environment for DB Api

[russleyshaw]

We need a place to develop tests to test against our API

Our non-API unit tests will probably be using mocha test framework with should.js assertions https://mochajs.org/ https://shouldjs.github.io/

However, for specific API end point testing we might want to use something more like chai & chaiHttp http://mherman.org/blog/2015/09/10/testing-node-js-with-mocha-and-chai/

This should be investigated by locally implementing a basic API test for a current endpoint (like POST /api/v2/user/) and reporting back to this issue. Once this has been investigated, Issue will be switched to development

[renodubois]

Hey, I know I haven't done much here but I'd really like to get some experience with both APIs and unit testing. Any way I could also get in on this?

[russleyshaw]

Sure. A good place to start is with the POST /api/v2/user/ endpoint. Typically testing involves normal test cases and robustness test cases.

A normal test case might be as simple as properly POSTing a new user.

A robustness test case might be like trying to POST a new user with an id given in the body (we dont want to post with an id because those are generated automatically.

One thing to note is since we are running everything in docker, the unit tests will need to as well. You can see in Colisee how we implemented tests in a docker environment

https://github.com/russleyshaw/Colisee/blob/master/test.sh

Basically, we spun up the docker containers, and then executed npm test directly on the container.

Typically it is also a good idea to clear out the database the tables before (and maybe after) starting a set of unit tests. In mocha you can add a before section and you can delete all elements in the user table using knex('user').del().asCallback...

Tests should be runnable with npm test by adding a test script to package.json. and all test files should exist in a test/ folder.

If chai/chaiHttp server doesnt work out, you can always make raw HTTP requests, but I would wrap those up to make testing easier.

[russleyshaw]

Also, tests are probably best in Javascript, not typescript

[russleyshaw]

Something to be aware of (since we are using react on Ophilia and Colisee) http://facebook.github.io/jest/

[russleyshaw]

This is pretty important.

[user404d]

I agree this is a serious issue and not having it will impede meaningful progress on other issues.

[russleyshaw]

Considering the priority of this, ill take it over for now and make individual issues one a framework has been set

[russleyshaw]

https://github.com/siggame/coliselia/tree/russley/api_test

@user404d @Daniel17sep @renodubois @jonsimington

I need a second pair of eyes on this. Tests pass locally (at least on my machine), but I cannot get it to work on travis

[russleyshaw]

Tests work locally but Travis CI is still being annoying

[renodubois]

Link to your branch is seemingly broken.

[user404d]

Unhandled rejection AssertionError: expected Error {
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '172.18.0.3',
  port: 5432,
  message: 'connect ECONNREFUSED 172.18.0.3:5432'
} not to be truthy (false negative fail)
    at Assertion.fail (/dbapi/node_modules/should/cjs/should.js:231:17)
    at Assertion.value [as ok] (/dbapi/node_modules/should/cjs/should.js:316:17)
    at knex.del.then.catch (/dbapi/test/user.js:28:32)
    at tryCatcher (/dbapi/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/dbapi/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (/dbapi/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/dbapi/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/dbapi/node_modules/bluebird/js/release/promise.js:687:18)
    at Async._drainQueue (/dbapi/node_modules/bluebird/js/release/async.js:138:16)
    at Async._drainQueues (/dbapi/node_modules/bluebird/js/release/async.js:148:10)
    at Immediate.Async.drainQueues (/dbapi/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:637:20)
    at tryOnImmediate (timers.js:610:5)
    at processImmediate [as _immediateCallback] (timers.js:582:5)

    1) "before all" hook: Clean up

Did you hard code the connection settings? They are going to be different on different machines potentially.

It seems like the environment variable of the process isn't getting the correct db hostname? I tried setting it to localhost but that didn't work either.

The issue is that the tests are being run before postgres can get up and running. I think the tests begin while docker is still propping up the services, so the connection fails because of this.

Perhaps a solution is to basically block and poll the postgres db until the connection can be made? Then have a timeout which will kill the test script if it fails to establish a connection.

[user404d]

Fixed with #20, but the solution should be discussed probably. See this addition.

[user404d]

Effectively the issue is that

docker-compose --file blah.yml up -d

sends the containers to the background to finish their startup procedures which effectively means that the tests can begin before the coliselia database can be created. The fix for this currently is to poll postgres and attempt to connect to the coliselia database and execute a simple command. If the command succeeds then the tests can proceed as coliselia is ready to accept queries in postgres.

[russleyshaw]

So a better solution IMO is to add a common test case at the beginning of each test that basically polls a couple times to make sure the db is up https://mochajs.org/#retry-tests

[russleyshaw]

but this is fine