search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.15k stars 5.47k forks source link

salt-ssh imports unrelated python modules on both slave (fatal) and master #7913

Closed pfalcon closed 9 years ago

pfalcon commented 11 years ago 
$ salt-ssh -c . localhost test.ping
localhost:
    ERROR: Permissions problem, target user may need to be root or use sudo:
     [CRITICAL] Unable to import msgpack or msgpack_pure python modules
    WARNING: '/etc/salt' directory does not exist.
    Failed to create directory path "/etc/salt/pki/minion" - [Errno 13] Permission denied: '/etc/salt'

msgpack or msgpack_pure should not be needed for salt-ssh.

Also, running with -lall:

Traceback (most recent call last):
  File "/home/pfalcon/projects-3rdparty/salt/.venv/local/lib/python2.7/site-packages/salt/loader.py", line 641, in gen_functions
    ), fn_, path, desc
  File "/home/pfalcon/projects-3rdparty/salt/.venv/local/lib/python2.7/site-packages/salt/log/handlers/logstash_mod.py", line 114, in <module>
    import zmq
  File "/usr/local/lib/python2.7/dist-packages/pyzmq-2.2.0.1-py2.7-linux-i686.egg/zmq/__init__.py", line 51, in <module>
    from zmq import core, devices
  File "/usr/local/lib/python2.7/dist-packages/pyzmq-2.2.0.1-py2.7-linux-i686.egg/zmq/core/__init__.py", line 26, in <module>
    from zmq.core import (constants, error, message, context,
ImportError: libzmq.so.3: cannot open shared object file: No such file or directory

I have msgpack installed in virtualenv, so in this case master doesn't error out, but otherwise it still does - but there shouldn't be a need for msgpack or zmq with salt-ssh.

basepi commented 11 years ago

Thanks for the report. We're still cleaning up little problems like this with salt-ssh, we'll get this taken care of.

What version are you using? There have been a lot of changes in salt-ssh since it was released in 0.17.0, and 0.17.1 was released yesterday.

pfalcon commented 11 years ago

I'm using git develop branch HEAD as of report time - sorry, with first reports, I added specific rev, but with bunch I filed, that piece got dropped. It was 0a31f3b38a3474f56523b99199be91eaf721e661

basepi commented 11 years ago

Cool. We'll get this fixed.

deftpunk commented 10 years ago

Using salt-ssh version 0.17.1 on a same version salt-master, some commands failed w/ the "Unable to import msgpack ..." message but some did not.

root@vwc-s36:/var/log/salt# salt-ssh '*' cmd.run 'dpkg -l |grep apt' vwc-vesx-dc1-77: [CRITICAL] Unable to import msgpack or msgpack_pure python modules Usage: salt-call [options] [arguments] salt-call: error: -l option requires an argument

b>root@vwc-s36:/var/log/salt#</b dpkg -l|grep salt ii salt-common 0.17.1-2precise Shared libraries that salt requires for all packages ii salt-master 0.17.1-2precise This package provides a remote manager to administer servers via salt ii salt-ssh 0.17.1-2precise remote manager to administer servers via salt

b>root@vwc-s36:/var/log/salt#</b pip search msgpack msgpack-python - MessagePack (de)serializer. INSTALLED: 0.1.10 LATEST: 0.4.0 msgpack-rpc-python - MessagePack RPC msgpack-pure - MessagePack (de)serializer written in pure Python. ....

b>root@vwc-s36:/var/log/salt#</b python Python 2.7.3 (default, Sep 26 2013, 20:03:06) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import msgpack import msgpack_pure Traceback (most recent call last): File "", line 1, in ImportError: No module named msgpack_pure

b>root@vwc-s36:/var/log/salt#</b salt-ssh '*' test.ping vwc-vesx-dc1-77: True

b>root@vwc-s36:/var/log/salt#</b cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=12.04 DISTRIB_CODENAME=precise DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS"

jeteokeeffe commented 10 years ago

Just a heads up .... also broken in 17.2 on CentOS 6.5 x86_64

[root@salt salt]# salt --versions Salt: 0.17.2 Python: 2.6.6 (r266:84292, Nov 22 2013, 12:16:22) Jinja2: 2.2.1 M2Crypto: 0.20.2 msgpack-python: 0.1.13 msgpack-pure: Not Installed pycrypto: 2.0.1 PyYAML: 3.10 PyZMQ: 2.2.0.1 ZMQ: 3.2.4

[root@salt salt]# salt-ssh test-server salt.minion test-server: [CRITICAL] Unable to import msgpack or msgpack_pure python modules

stephankaag commented 10 years ago

Any news?

Gehn commented 10 years ago

Still an issue for me as well, CentOS 6.2 minion and master, salt version 0.14.1

davidjb commented 10 years ago

This is still an issue for me; RHEL 7 minion, Salt version 2014.1.11. Can't test with newer development Salt versions due to another issue affecting salt-ssh.

thatch45 commented 10 years ago

The current dev version of Salt should have this resolved, I am looking into the other mentioned issue

thatch45 commented 9 years ago

This has been resolved in 2014.7

bbinet commented 9 years ago

This issue is still there with 2014.7.0:

bruno@neo: $ mktmpenv 
New python executable in fb282f143b5a1ff0/bin/python
Please make sure you remove any previous custom paths from your /home/bruno/.pydistutils.cfg file.
Installing setuptools.............done.
Installing pip................done.
This is a temporary environment. It will be deleted when you run 'deactivate'.

(fb282f143b5a1ff0)bruno@neo: $ pip install salt-ssh
Downloading/unpacking salt-ssh
  Downloading salt-ssh-2014.7.0.tar.gz (3.5MB): 3.5MB downloaded
  Running setup.py egg_info for package salt-ssh
    2014.7.0

Downloading/unpacking Jinja2 (from salt-ssh)
  Downloading Jinja2-2.7.3.tar.gz (378kB): 378kB downloaded
  Running setup.py egg_info for package Jinja2

    warning: no files found matching '*' under directory 'custom_fixers'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
Downloading/unpacking msgpack-python>0.3 (from salt-ssh)
  Downloading msgpack-python-0.4.2.tar.gz (114kB): 114kB downloaded
  Running setup.py egg_info for package msgpack-python

Downloading/unpacking PyYAML (from salt-ssh)
  Downloading PyYAML-3.11.tar.gz (248kB): 248kB downloaded
  Running setup.py egg_info for package PyYAML

Downloading/unpacking MarkupSafe (from salt-ssh)
  Downloading MarkupSafe-0.23.tar.gz
  Running setup.py egg_info for package MarkupSafe

Downloading/unpacking requests>=1.0.0 (from salt-ssh)
  Downloading requests-2.5.0.tar.gz (443kB): 443kB downloaded
  Running setup.py egg_info for package requests

Installing collected packages: salt-ssh, Jinja2, msgpack-python, PyYAML, MarkupSafe, requests

[...]
jcftang commented 9 years ago

I'm also experiencing this issue with salt-ssh 

# salt-ssh 'node0100' cmd.run uptime
node0100:
    ----------
    retcode:
        1
    stderr:
        [CRITICAL] Unable to import msgpack or msgpack_pure python modules
        Traceback (most recent call last):
          File "/tmp/.root_6cb53b__salt/salt-call", line 4, in 
            salt_call()
          File "/tmp/.root_6cb53b__salt/salt/scripts.py", line 122, in salt_call
            client.run()
          File "/tmp/.root_6cb53b__salt/salt/cli/__init__.py", line 419, in run
            caller.run()
          File "/tmp/.root_6cb53b__salt/salt/cli/caller.py", line 225, in run
            ret = self.call()
          File "/tmp/.root_6cb53b__salt/salt/cli/caller.py", line 129, in call
            ret['return'] = func(*args, **kwargs)
          File "/tmp/.root_6cb53b__salt/salt/modules/cmdmod.py", line 623, in run
            fn_.write(serial.dumps(jid_dict))
          File "/tmp/.root_6cb53b__salt/salt/payload.py", line 115, in dumps
            return msgpack.dumps(msg)
        NameError: global name 'msgpack' is not defined
    stdout:

With this ...

# salt --versions-report
           Salt: 2014.7.0
         Python: 2.7.3 (default, Dec 18 2014, 19:10:20)
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
 msgpack-python: 0.3.0
   msgpack-pure: 0.1.3
       pycrypto: 2.4.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 13.0.0
           RAET: Not Installed
            ZMQ: 3.2.2
           Mako: Not Installed
basepi commented 9 years ago

Thanks for the reports. I'm going to re-open this and find some time to track down these msgpack imports.

jcftang commented 9 years ago

@basepi I think I figured out what my problem is with salt-ssh for my case... Here's an example of the problem when I try running salt-call on the target

/tmp/.root_6cb53b__salt# python
Python 2.7.3 (default, Mar 14 2014, 17:55:54) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>  msgpack.loads(msgpack.dumps([1, 2, 3]), use_list=True)
  File "", line 1
    msgpack.loads(msgpack.dumps([1, 2, 3]), use_list=True)
    ^
IndentationError: unexpected indent
>>> import msgpack
Traceback (most recent call last):
  File "", line 1, in 
  File "msgpack/__init__.py", line 3, in 
    from msgpack._msgpack import *
ImportError: msgpack/_msgpack.so: wrong ELF class: ELFCLASS64

Basically I am launching salt-ssh from my intel x86_64 based machine and salt-ssh is copying out some intel compiled objects to my target which is actually a debian armhf based machine. I guess the correct solution is check for the architecture of the target if its the same as the host running salt-ssh and run a python only version of msgpack if it's not the same architecture.

This problem probably affects hosts that are intending on running salt-ssh from a 64bit master targetting a 32bit minion.

basepi commented 9 years ago

msgpack should actually not be required at all for salt-ssh. I think we're still including it in the thin just for safety, but what we really need to do is track down any ungated calls to msgpack and gate them for salt-ssh.

@jcftang @bbinet Can anyone reproduce any msgpack errors with the patch in #19742? It should be safe to remove msgpack completely from the thin, but I want to see if we can catch any bad imports of msgpack before we merge that.

jcftang commented 9 years ago

@basepi I can confirm that #19742 resolves the above for me, I did have to clear out the thin cached tarball and also /tmp on the target.

~# salt-ssh 'node0100' cmd.run uptime
node0100:
     12:06:58 up 156 days,  2:02,  1 user,  load average: 0.08, 0.05, 0.05

It at least works for cmd.run in the above simple test.

basepi commented 9 years ago

Thanks for the update, @jcftang. I'm going to close this again for now. Please comment if you see any msgpack errors in salt-ssh going forward, they should be quick and easy to fix if we've missed any.

mventimi commented 9 years ago

I'm seeing the same error when I bootstrap from git develop:

$ salt-ssh -i lab cmd.run 'uname -a'
lab:
    ----------
    retcode:
        1
    stderr:
        [CRITICAL] Unable to import msgpack or msgpack_pure python modules
        Traceback (most recent call last):
          File "/tmp/.pi_6787e8__salt/salt-call", line 4, in <module>
            salt_call()
          File "/tmp/.pi_6787e8__salt/salt/scripts.py", line 220, in salt_call
            client.run()
          File "/tmp/.pi_6787e8__salt/salt/cli/call.py", line 50, in run
            caller.run()
          File "/tmp/.pi_6787e8__salt/salt/cli/caller.py", line 235, in run
            ret = self.call()
          File "/tmp/.pi_6787e8__salt/salt/cli/caller.py", line 139, in call
            ret['return'] = func(*args, **kwargs)
          File "/tmp/.pi_6787e8__salt/salt/modules/cmdmod.py", line 617, in run
            jid_dict = serial.load(fn_)
          File "/tmp/.pi_6787e8__salt/salt/payload.py", line 112, in load
            return self.loads(data)
          File "/tmp/.pi_6787e8__salt/salt/payload.py", line 97, in loads
            return msgpack.loads(msg, use_list=True)
        NameError: global name 'msgpack' is not defined
    stdout:

$ salt --versions-report
           Salt: 2015.2.0-307-g1fc9a52
         Python: 2.6.9 (unknown, Sep 13 2014, 00:25:11)
         Jinja2: 2.7.2
       M2Crypto: 0.20.2
 msgpack-python: 0.4.5
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 3.2.4
           Mako: Not Installed

Could this be a regression? It doesn't happen with 2014.7.1:

$ salt-ssh -i lab cmd.run 'uname -a'
lab:
    Linux raspberrypi 3.12.28+ #709 PREEMPT Mon Sep 8 15:28:00 BST 2014 armv6l GNU/Linux

$ salt --versions-report
           Salt: 2014.7.1
         Python: 2.6.9 (unknown, Sep 13 2014, 00:25:11)
         Jinja2: 2.7.2
       M2Crypto: 0.20.2
 msgpack-python: 0.4.5
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 3.2.4
           Mako: Not Installed
basepi commented 9 years ago

@mventimi Thanks for the update. Would you mind opening a new issue for this one? Just include all the information you've included here.

basepi commented 9 years ago

@mventimi Yes, I've done a little bit of investigation into this and this is related to some changes made in develop. More to come when I get a chance to talk to @thatch45 next week, but we should definitely open a new issue.

mventimi commented 9 years ago

Thanks @basepi. Issue #20276 opened.