Including Hub Server for integration tests

th0mas commented 4 years ago

To be able to properly run integration tests against the dwyl/smart-home-* stack we should include the Hub server in our test environment so we can test against it.

This has several benefits:

No need to mock a hub client. We can use our actual hub client and run tests to make sure this is functioning as expected. This avoids any errors with our Mock implementation and makes the tests less complex to maintain.
Test the internal API. By testing against our actual hub server, we have the added benefit of checking our internal websocket api still works and we haven't accidentally created any breaking changes.
Integration tests across the whole stack. We can make sure our entire stack functions as expected by running integration tests that encompass the whole stack.

TODO:

[ ] Add Hub dependencies to travis - postgresql, dwyl_auth key etc.
[ ] Add hub server to travis through git(?)
[ ] Rewrite existing unit tests to use the real hub not our mock client
[ ] Write thorough integration tests across the whole stack.

th0mas commented 4 years ago

Notes:

We don't have to run our hub as a separate process -- we can import the Phoenix endpoint and run in the same BEAM instance - this will give us more control over the tests. It does, however, tie the tests together and means the projects can no longer be tested "separately"

th0mas commented 4 years ago

We need to remember to give our install scripts executable permissions with chmod +x $SCRIPT.sh

th0mas commented 4 years ago

Installing the hub server on travis results in a hang after database migrations:

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received

I'm not sure why We use exactly the same install process for the hub server tests and that works

th0mas commented 4 years ago

We need to run the hub server as a background process: mix phx.server &.

This means that travis doesn't exit however as it waits for background processes to finish first...

th0mas commented 4 years ago

Caching our hub deps breaks git clone as you must clone into an empty directory.

To get around this I'm copying the cached deps in and out of a temporary directory and runtime. E.g. to cache the dependencies at the end of the build process:

- mkdir smart-home-auth-server-deps
- cp -R smart-home-auth-server/deps/* smart-home-auth-server-deps

th0mas commented 4 years ago

creating directory /home/travis/build/dwyl/smart-home-firmware/smart-home-auth-server-deps
...
cp: cannot stat 'smart-home-auth-server-deps/*': No such file or directory

what

th0mas commented 4 years ago

The hub client process can't connect on travis, i'm not sure why as all ports etc should be the same....

th0mas commented 4 years ago

Going to try and read our hub logs with tail -f /proc/$SERVER_PID/fd/1

th0mas commented 4 years ago

Multi-container workflows seem to be supported on Circle Ci....

https://circleci.com/blog/setting-up-tricky-containers-in-circle-2-0-multi-image/

th0mas commented 4 years ago

Worked around with dwyl/smart-home-auth-server#14

dwyl / smart-home-firmware

Including Hub Server for integration tests #13

TODO:

Notes: