search

taurus-org / taurus

Moved to https://gitlab.com/taurus-org/taurus
http://taurus-scada.org
43 stars 46 forks source link

CI showing random segfaults #1042

Closed cpascual closed 4 years ago

cpascual commented 4 years ago

I am concerned about some segfaults that we are lately seeing in the tests . The traces seem to blame (py?)tango's event subscription (tox reports using pytango 9.3.1 from conda).

The segfault is not happening all the time, but it is worrying enough.

Hint: If we know of some some specific version of (py)tango (or other related lib) that is not affected, maybe we could try to pin the safe version in the tox.ini

cpascual commented 4 years ago

Now that tango-controls/cppTango#699 is merged, we expect that some of the instability problems in our CI can be solved if we use cppTango >= 9.3.4rc5 in our CI.

Currently we pinned 9.3.3 in travis.

Note: The tango docker images from https://github.com/tango-controls/tango-cs-docker seem to be no longer updated. Therefore we could consider moving to the docker images produced by SKA: https://github.com/ska-telescope/ska-docker

cpascual commented 4 years ago

Note to myself: check the possibility that the paralellization of the jobs is causing errors (maybe because different parallel jobs use the same tango network name?)

This idea comes from the following note here:

Never use the network=host mode when running CI jobs, and add extra distinguishing identifiers to the network name and container names to prevent collisions with jobs from the same project running at the same time.

*UPDATE: this seems not to be the case

cpascual commented 4 years ago

1109 seems to fix this