Fix travis builds

[cpascual]

(Edited and updated)

Try various approaches to fix #1042 :

• Use an unique docker project name per build to test (see this)
• Use newer docker images (with a composer configuration adapted from SKA's)
• Avoid using Tango for TaurusValueTest.test_text (use a writable eval attribute instead of a tango one)
• Allow failures for python2 tests in Travis (they hang/crash on exit of the pytest command, but typically the testsuite finishes without failures)

[cpascual]

The usage of different project names does not seem to have effect. But the replacement of the tango-cs docker images by the newer ones from SKA does seem to improve the situation.

In this pipeline we can see that not a single job shows signs of crashed tango DS (before it was very common). But there are still some issues:

• The py2-qt5 job now shows the same problem as the py2-qt4 (the test suite finishes but the process does not exit and gets killed by Travis)
• The py36-qt5 job seems to systematically fail in test taurus.qt.qtgui.panel.test.test_taurusvalue.TaurusValueTest testMethod::test_texts

I've managed to reproduce both issues locally using tox, but they happen a lot less than in travis.

Note that the py36-qt5 issue is not related with this change of docker images. It also happened with the tango-cs images

...some more work needed

[cpascual]

Regarding the timeouts, maybe we could use pytest-timeout to get some more debugging info

[cpascual]

Hi @reszelaz , can you please review this and eventually merge so that we can properly test the other PRs?

The things that worked were:

• Using newer Tango images from SKA (tango v9.3.3)

• I changed the TaurusValue tests to reduce the probability of tango issues with the texts test (we saw them recurrently, and specially in the py36-qt5 target.

• I also set the py2 tests as "allowed to fail" in Travis because of the crashes/hangs on pytest exit (note that they can still be useful for local run until we fix them or drop support for python2)

Still missing (but can be done out of this PR):

• use tango v>=9.3.4-rc5 for both the local pytango client and the tango services

[reszelaz]

But IIRC in TangoSchemeTest there are some events pushed but I don't remember if we use them in the tests.

I think that TangoSchemeTest should not be problematic - this one runs on the host and not in the docker container.

[cpascual]

I assume that the docker compose implementation (Makefile and yaml) come from the SKA's project, as they are, right? So, no need to review them. I have tried them locally (CI_JOB_ID=local make start tangotest) and these work as expected.

I did only some minor changes to SKA's files:

• I linked the databaseds port 10000 to the host's 10000 (50a7dcc)
• I adapted it to use Travis job ID (3cae5ad)
• I removed unused machines (itango, astor, archiver,...) (4b5c81f)

BTW I think the jive.yml is not strictly necessary?

No, but as I mentioned in the 4b5c81f, it does not harm because it shares the image with tangotest, and instead it helps a lot for debugging locally (specially on a Debian machine where Jive is not available as an official deb package)

Just a note, I have not managed to receive events on my host from the docker containers i.e. I receive API_EventTimeout. And I think that the testsuite run on the host on Travis, righ? I imagine this is normal cause we don't open the ZMQ ports (which are unknown beforehand). Or maybe I miss something.

AFAIK, It is not about ports being open, but about network separation. The containers are using a bridge network which is isolated from the host network. The containers can see each other fully and exchange events between them, but not with the host (the host may reach the containers by IP, but not by name). On top of that, we do the link of databaseds:10000 -- host:10000, so that the databaseds really serves both the host network and the containers network.

Therefore if we really need to do tests involving events, there are some alternatives:

1. we can spawn the required DS in the host network (this is what we do with TangoSchemeTest as you already mentioned)
2. We can spawn the DS as a container and also the client as a container (so that both are in the container network)
3. we can try to use a different network configuration in the composer (maybe using a single network)

For the moment, I'd say that option 1 is enough for us

Also, maybe as an idea to evaluate in the future, if we would like to base the tests on TangoSchemeTest DS only then we could think of refactoring the tests to use tango.test_context.DeviceTestContext. Then we wouldn't need the database, tests will be much easier to run for developers and this will automatically open testing on Windows.

Yes, this has been mentioned before (I even thought there was an open issue, but I could not find it). I agree that this would simplify the current testsuite and allow for better unit tests. But still, one thing to note is that with docker-compose we can potentially do more complex and closer-to-production functional tests than with DeviceTestContext (and it also allows testing on Windows since the tango server infrastructure is run on containers)

[reszelaz]

I should have read the particular commits messages, this would have saved some time:)

Yes, I actually wanted to mean expose ports when I said open ports, sorry if it was not clear.

Regading the testing on Windows I was referring to CI wtih Appveyor. I based this on my attempt to configure Appveyor by re-using as much as possible our already existing Docker images. I remeber that it was not possible without going for Pro accounts.. But this was a year ago, so things may have changed.