Open jacobEAdamson opened 5 years ago
Thanks, from @jacobEAdamson comments in the pull request and seeing here you mention python 3.7 things are a little more clear, and a little more confused.
self.encode()
is going to return a bytes object on python3 which will mean the following
$ python
Python 2.7.16 (default, Apr 30 2019, 15:54:43)
>>> subprocess.list2cmdline(b"testing 123")
't e s t i n g " " 1 2 3'
$ python3
Python 3.7.4 (default, Jul 9 2019, 16:32:37)
>>> subprocess.list2cmdline(b'testing 123')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.7/subprocess.py", line 530, in list2cmdline
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'int' is not iterable
So that explains the error above, but this raises more questions for me ...
It looks like this happened originally with 4ab331a82972d5dc7e77c7078fb7847e638bf5be which gives the clue it was related to something like ISO-8859-15 systems and a command like
Command("ls -l %s", "/é")
I think maybe the problem was that you're calling Popen("ssh remotehost ls -l /é") and so Popen encodes the "/é" for the local system (https://github.com/python/cpython/blob/v3.7.3/Lib/subprocess.py#L1436) but then that is incorrect when the remote is running something like ISO-8859-15, and it now gets a (say) UTF-8 encoded command-line to try and run? So by making it a bytes-like object you short-circuit python trying to encode for you?
Then there's a bunch of suff about CreateProcessA and CreateProcessW on windows that suggests that running commands should either be an ascii string or UTF-16 chars. Then I just get lost ... :)
Same problem. Only python 2.7 seems to work. I've tried 3.5 and 3.7 as well and got the above mentioned: needquote = (" " in arg) or ("\t" in arg) or not arg TypeError: argument of type 'int' is not iterable_
Similar problem. Windows 2008 r2 + Python 3.8.3 + testinfra 5.2.1.
I got the error bytes args is not allowed on Windows
when trying to use run_local
:
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 b
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import testinfra
>>> host = testinfra.host.get_host('local://')
>>> host.run("echo")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\
ackages\testinfra\host.py", line 75, in run
return self.backend.run(command, *args, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\
ackages\testinfra\backend\local.py", line 30, in run
return self.run_local(self.get_command(command, *args))
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\
ackages\testinfra\backend\base.py", line 192, in run_local
p = subprocess.Popen(
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\
cess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\
cess.py", line 1239, in _execute_child
raise TypeError('bytes args is not allowed on Windows')
TypeError: bytes args is not allowed on Windows
The workaround for me is a runtime method patching removing the self.encode(command)
.
Hey Vinicius,
The workaround for me is a runtime method patching removing the
self.encode(command)
.
Can you show me how you did this? I've tried a lot of things now, but I did not succeed. This is my test_infra.py file:
import sys
sys.path.append('C:\\Python39\\Lib\\site-packages\\testinfra\\backend\\')
from base import BaseBackend
def run_local_new(self, command, *args):
command = self.quote(command, *args)
p = subprocess.Popen(
command, shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
stdout, stderr = p.communicate()
result = self.result(p.returncode, command, stdout, stderr)
return result
BaseBackend.run_local = run_local_new
def test_puppet_facts(host):
facts = host.facter()
assert facts["operatingsystem"] == "windows"
But when running the test via
C:\Python39\python -m pytest -v .\test_infra.py
I still get
C:\Python39\lib\site-packages\testinfra\modules\puppet.py:105: in __call__
return json.loads(self.check_output(cmd))
c:\python39\lib\site-packages\testinfra\host.py:75: in run
return self.backend.run(command, *args, **kwargs)
c:\python39\lib\site-packages\testinfra\backend\local.py:30: in run
return self.run_local(self.get_command(command, *args))
C:\Python39\lib\site-packages\testinfra\backend\base.py:192: in run_local
p = subprocess.Popen(
C:\Python39\lib\subprocess.py:947: in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
[...]
TypeError: bytes args is not allowed on Windows
How did you patch the run_local(self, command, *args)
method and how can I do this with the approach above? For me it looks like as if my patched run_local method is not used.
How did you patch the
run_local(self, command, *args)
method and how can I do this with the approach above? For me it looks like as if my patched run_local method is not used.
In my conftest.py:
def patch_testinfra():
def run_local_new(self, command, *args):
command = self.quote(command, *args)
p = subprocess.Popen(
command, shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
stdout, stderr = p.communicate()
result = self.result(p.returncode, command, stdout, stderr)
return result
from testinfra.backend.base import BaseBackend
BaseBackend.run_local = run_local_new
def pytest_generate_tests(metafunc):
patch_testinfra()
@viniciusartur solution wasn't suffisent in my case (I'm using testinfra with docker backend), but combining its solution with https://github.com/pytest-dev/pytest-testinfra/issues/375#issue-360991022 made it work :
# conftest.py
import os
import re
import subprocess
import testinfra
def patch_testinfra():
if os.name == 'nt':
"""
Making testinfa work from Windows with Docker backend.
Inspired from
- https://github.com/pytest-dev/pytest-testinfra/issues/411#issuecomment-782729362
- https://github.com/pytest-dev/pytest-testinfra/issues/375#issue-360991022
"""
def quote(command, *args):
def anon_1(arg):
if re.match(r'/("|\s|\')', arg) != None:
return arg
arg = re.sub('"', '\"', arg)
return '"%s"' % (arg)
if args:
return command % tuple(anon_1(a) for a in args)
# return command % tuple(pipes.quote(a) for a in args)
return command
def run_local_new(self, command, *args):
command = quote(command, *args)
p = subprocess.Popen(
command,
shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
stdout, stderr = p.communicate()
result = self.result(p.returncode, command, stdout, stderr)
return result
from testinfra.backend.base import BaseBackend
BaseBackend.run_local = run_local_new
def pytest_generate_tests(metafunc):
patch_testinfra()
My system is Python 3.7.0 running Windows 10 and I receive an error when I try to run anything with run_local on windows (which affects all backends). The stack trace looks like this:
Looks like the fact that the command is being encoded before running is to blame. Popen in Windows doesn't handle byte arrays the same way it does on Linux. Was able to fix by modifying code to this:
Basically only encode when not on windows