UCL-CloudLabs / user-portal

Front end user portal for CloudLabs
0 stars 0 forks source link

Run tests with pytest xdist #73

Open raquelalegre opened 6 years ago

raquelalegre commented 6 years ago

Since we are now launching several VMs on Azure as part of the tests, the tests take much longer. It's easy to run the tests in parallel with pytest-xdist and it was just requires:

pip install pytest-xdist
pytest -n <num threads>

However it doesn't output any log and the "-s" option is no use here. I do find the log necessary just to check what terraform is doing, so I'm not going to commit this until I find a workaround. Someone in StackOverflow said this, which might help:

https://stackoverflow.com/questions/36726461/how-to-print-output-when-using-pytest-with-xdist?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

raquelalegre commented 6 years ago

This might start being a requirement for travis at least.

I am now testing 2 different machine types and 3 different UbuntuServer versions, which makes 6 deployment tests. They take locally 50 minutes.

There are also two problems with Travis, one the log is becoming too long to be handled by their interface and can only be seen by downloading the raw log. The other problem is that the testing time exceeds the maximum time Travis tests can be run.

For now I'll just run the tests in the Standard machine so there's fewer.

raquelalegre commented 6 years ago

We can also make the test more simple and just launch the machines, but don't deploy the whole Levine server. We can do the last bit on a different tests just once per test set.