Allow adapters to run test suite with their db

[mxstbr]

This adds a simple script that makes it possible to run our own test suite against a newly written adapter. Usage would be like so:

Testing your adapter

micro-analytics contains an test suite to verify everything works as expected. We also export this test suite to make it possible to verify your adapter works as expected!

To run our test suite with your adapter (we recommend doing this in CI too) you only have to do three steps:

First, npm install --save-dev micro-analytics-cli to get the test suite. Second, create a new file called test.js which contains this code: (don't forget to replace './index' with the relative path to your main entry file!)

#!/usr/bin/env node
const test = require('micro-analytics-cli/adapter-test')

test('./index')

Finally, run our test suite with your db adapter with node test.js! 🎉

---

@relekang I've published a pre-release version containing this change under micro-analytics-cli@dev. Mind trying this on your micro-analytics-adapter-redis and seeing if it works?

Closes #19

[codecov-io]

Codecov Report

Merging #31 into master will decrease coverage by 1.81%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #31      +/-   ##
==========================================
- Coverage   83.63%   81.81%   -1.82%     
==========================================
  Files           3        3              
  Lines          55       55              
  Branches       13       12       -1     
==========================================
- Hits           46       45       -1     
- Misses          9       10       +1

Impacted Files	Coverage Δ
src/db.js	60% <0%> (-10%)	:arrow_down:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c6b1c44...6665cb5. Read the comment docs.

[mxstbr]

Hm, I just tried this for the flat-file-db adapter and Jest complains:

No tests found
No files found in /path/micro-analytics-adapter-flat-file-db/node_modules/micro-analytics-cli.
Make sure Jest's configuration does not exclude this directory.
To set up Jest, make sure a package.json file exists.
Jest Documentation: facebook.github.io/jest/docs/configuration.html

The tests exist though, everything is exactly where it needs to be... Any ideas what's going wrong here?

[mxstbr]

This is a bug upstream in jest-haste-map according to @cpojer, trying to get a fix in there!

[mxstbr]

Okay I've managed to circumvent the bug with a super ugly and slow, but working, hack.

I npm link the adapter, clone the whole micro-analytics project to /tmp, install the dependencies, run the tests with the adapter and then delete the temp folder again.

I've verified this works with the flat-file-db adapter. @relekang would you mind giving this a try?

[relekang]

This looks neat 🙂

I did not get it to run on the redis adapter 😕

> micro-analytics-cli@1.1.0 test /private/tmp/micro-analytics-adapter-test-suite
> jest

 FAIL  tests/atomicity.test.js
  ● Test suite failed to run

    ProcessTerminatedError: cancel after 2 retries!

      at Farm.<anonymous> (node_modules/worker-farm/lib/farm.js:81:25)
      at Array.forEach (native)
      at Farm.<anonymous> (node_modules/worker-farm/lib/farm.js:75:36)
      at ontimeout (timers.js:365:14)
      at tryOnTimeout (timers.js:237:5)
      at Timer.listOnTimeout (timers.js:207:5)

A worker process has quit unexpectedly! Most likely this an initialization error.
npm ERR! Test failed.  See above for more details.
~/dev/micro-analytics-adapter-redis

[mxstbr]

Hmm, will take a look later!

[mxstbr]

Picking this back up, switching to mocha instead of Jest has resolved the "Test can't run" error. Now working on making it a solid test suite, should be good to go after I'm done!

[mxstbr]

Okay this works now. Slightly changed instructions, but it's largely the same! 🎉

The tests of the main repo now test the main repo, but they can also be used to test any third party adapter. I'll do the redis adapter now to showcase how it works!

Note: Except the flat-file-db one—that will be a bit tricky, since that's also required inside micro-analytics, not sure if I can get that tested just yet. Will work on that next!

[mxstbr]

Damn this is still failing due to babel-register. Will get it sorted soon! We're really close now.

[relekang]

🤔 Yeah, probably because we don't get the dev dependencies of a dependency. Adding the necessary dependencies in the adapter might be a solution? or is that what the npm i in adapter.test.js is about?

[mxstbr]

I'm npm installing them actually, see adapter-test.js, but babel can't find the preset because it looks from the root cwd instead of the __dirname. 😕

[jimthedev]

@I appreciate the attempt to use Node for this but it might be easier to keep node out of the equation when triggering the tests to be run. The invocation seems to work if you do it manually and don't try to use require.

What if an adapter could just list micro-analytics-cli as a node devDependency then trigger the tests via bash/npm script. The micro-analytics-cli would first npm install the dependencies but then would manually copy the adapter from the cwd into the child node_modules folder of the cli. The idea here is that the invocation of node doesn't happen until the dependency structure has been setup using just bash. The bash command would look something like node node_modules/micro-analytics-cli/adapter-test.js micro-analytics-adapter-redis and then adapter-test.js could grab either a npm module name or a path from process.argv.

[mxstbr]

If you think that could work out please take a stab at it! I'm really stuck here.

[jimthedev]

I'll see what I can scrape together.

[relekang]

I think we also might need an adapter test suite that has a few basic unit tests, since a test that fails for the micro-analytics project might be hard to track down for people who are not very familiar with the codebase. What I am suggestion is to have micro-analytics-cli/adapter-tests/unit-tests.js and micro-analytics-cli/adapter-tests/integration-tests.js. I have started on the unit tests.

What do you think?

[mxstbr]

Sure!

[relekang]

Closing this as I think we need to start over if we want to revisit this after the #69. However, I think the approach of writing specific tests for adapters works nicely so far.