Instalation Salt from official satltstack repository changes major version of python-tornado on Ubuntu LTS

[es1o]

Hello.

Why saltstack package on Ubuntu require python-tornado with different major version than python-tornado from official ubuntu repository.

I'm using Ubuntu 14.04 LTS and after salt installation my python-tornado was upgraded from 3.3.1 to 4.2.1 and my Python application stopped work.

[jfindlay]

@es1o, starting with 2015.8.1, salt has dependencies on tornado features only available in version 4.2.1 and later.

[es1o]

Ok I understand, but why salt broke and modify my system? I use LTS because I want to have rock solid and stable environment for my application. I think this is a serious issue.

[jfindlay]

The best I can suggest, then is to use the 2015.5 branch of salt.

[es1o]

But it doesn't solve my problem because system installs wrong python-tornado version.

Only removing wrong python-tornado package from saltstack repository for 14.04 solve my problem.

[godfryd]

This is a real problem. We just cannot use Salt and Tornado at the same time. It starts to be insane if I have some Salt state that requires tornado=3.3.1 or any other app or library that at the same time is a dependency of Salt. It will just not work. Maybe salt should embed its private version of tornado and its other dependencies.

[dmurphy18]

@godfryd Having our own version of python-tornado would introduce other issues since different packages would be trying to install / control the same files. This would cause errors. The selection would have to be either use the installed python-tornado or if none installed then use salt's version of python-tornado. Even if we change the label to python-tornado-salt, we would still have an issue outlined in the first sentence.

Your installed version of python-tornado is insufficient to use any release from the 2015.8 branch of salt, which requires at a minimum v4.2.1 (v4.2 had a CVE).

Hence jfindlay's suggestion of 2015.5 branch releases where there is no dependency on python-tornado is the best that can be achieved.

[godfryd]

I understand your point. I could not use older version of Salt as it has bugs. Eventually we had to replace Salt with other similar tool as this was blocking issue for us.

[es1o]

Hence jfindlay's suggestion of 2015.5 branch releases where there is no dependency on python-tornado is the best that can be achieved.

I can't because I can't add official salt stack repository to my system because package from this repository broke my system. If I install python-tornado, Ubuntu will install your version from your repository and replace my stable python-tornado package version from LTS and broke my app. In my opinion this is a serious problem because I can't use Salt to manage my infrastructure because this problem exists on stable RHEL, CentOS, Debian and Ubuntu systems.

[damon-atkins]

One thought is why does the salt minion need to use python-tornado? (other than consistency) "Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user."

Would have thought the minion only needs a few connections?

However I assume something being used in python-tornado 3.x is deprecated in python-tornado 4.x of tornado eventual the software using 3.x will need to be updated not to use deprecated methods/functions

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[es1o]

This issue still valid, please do not close.

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.

[damon-atkins]

Question still stands as to why minion needs to use tornado?

OS Vendors Fix CVE issues, however the version number will not change, as they do their own patches. Salt should only increase the version required when functionality is required not due to CVEs.

[es1o]

@damon-atkins I think minion doesn't need tornado. Tornado package exists in saltstack repository in different version than in distribution. When I type apt-get upgrade, apt will download tornado from saltstack repository. This is a real problem and huge bug in my opinion. It is the same situation, that some project would have different kernel version in repository.

[damon-atkins]

RHEL salt-minion is dependant on salt which is dependant on python-tornado which is used in

/usr/lib/python2.7/site-packages/salt/client/__init__.py
/usr/lib/python2.7/site-packages/salt/client/mixins.py
/usr/lib/python2.7/site-packages/salt/crypt.py
/usr/lib/python2.7/site-packages/salt/engines/ircbot.py
/usr/lib/python2.7/site-packages/salt/engines/webhook.py
/usr/lib/python2.7/site-packages/salt/master.py
/usr/lib/python2.7/site-packages/salt/minion.py
/usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/__init__.py
/usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.py
/usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado_websockets.py
/usr/lib/python2.7/site-packages/salt/pillar/__init__.py
/usr/lib/python2.7/site-packages/salt/transport/ipc.py
/usr/lib/python2.7/site-packages/salt/transport/mixins/auth.py
/usr/lib/python2.7/site-packages/salt/transport/tcp.py
/usr/lib/python2.7/site-packages/salt/transport/zeromq.py
/usr/lib/python2.7/site-packages/salt/utils/thin.py
/usr/lib/python2.7/site-packages/salt/utils/async.py
/usr/lib/python2.7/site-packages/salt/utils/event.py
/usr/lib/python2.7/site-packages/salt/utils/http.py

Looks like its used a lot. Client might be able to get away with an older version?

[damon-atkins]

@es1o Make you own repo leave tornado out of the minion repo. Then the upgrade will not pull it down. However it does want v4.2.1. I found code which checks for =>4.0. Ubuntu 14.04 has 12 months of support left.

[es1o]

@damon-atkins Ubuntu has support now and I don't want to break my whole OS because salt repo contains package that override system package. In my opinion salt should have private tornado because that problem would occur in future.

[dmurphy18]

@damon-atkins If TCP support is selected in the Salt-minion configuration, then python-tornado is used rather than zeromq.

[damon-atkins]

@dmurphy18 Thanks for the info. Can python-tornado be removed as a requirement for the minion. And zeromq be removed as a requirement for the minion. Then people who have zeromq apps can use TCP and people who have tornado apps can use zeromq. And salt just errors if the required functionality is missing by using hasattr(). A better method would be to test for the used set of functions/methods instead of a version of a module. This relates to the minion, and not the master.

Side Note: Vendors patch the software, with fixes and CVE, rather than updating the software from the open source, Which means version 1.2.3 is not the same is open source version 1.2.3. They do this to keep backwards compatibility.

Have salt packages in one repo and 3rd part dependency in a different repo, would prevent the yum update issue. However people should make their own mirror/repo.

[dmurphy18]

@damon-atkins Currently python-tornado and zeromq are requirements for salt-minion. Having Salt detect the functionality and install appropriate packages would complicate matters, for example: on Debian, on Redhat, on Solaris, on AIX, etc. it would tend to hard bake platforms into Salt, being architecture independent is a better solution.

Note, there are methods whereby abstraction could make that process easier. but again complicating QA and environment.

Understand that Vendors patch software with fixes and CVE, however some Vendors do not respond and update packages in a timely manner, and do not update packages with versions required for Salt to function, esp. as platform releases near their end of life. For example: close to the end of RHEL 5's life, most dependent packages were supplied by SaltStack since support had been dropped for them, an example of this would be support for jinja.

However, I shall raise your issues internally about the dependencies, and the possibility of a dependencies repo.

[dmurphy18]

@es1o I would like to close this issue since Ubuntu 14.04 is about to reach EOL at the end of April. With Ubuntu 16.04 and 18.04, Salt now relies on the OS supplied version of python-tornado and steps have been taken to minimize the number of SaltStack supplied dependencies.

For example: with Ubuntu 18.04, Salt only requires dependencies for python-ioflo, python-raet which are deprecated and shall be removed in the Neon Release, approximately Fall this year, and python-timelib which will also be removed at that time, only leaving the Salt package. With Ubuntu 16.04 the dependencies apart from the previous ioflo and raet, are python-libcloud and python-libnacl.

If this resolves the issue for you, please consider closing this and sorry it took a while to remove the number of dependencies.

[es1o]

@dmurphy18 You can close, I stopped using saltstack 3 years ago because of this bug. ;)

[dmurphy18]

@es1o Sorry to hear that you stopped using Salt due to this issue. FYI - the later versions of Salt have minimized the dependencies provided by Salt to where it is essentially Salt and transport Raet and Raet is deprecated and will be removed in the Neon Release (scheduled for fall/end of year), for Ubuntu 16.04 and 18.04