search

pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.91k stars 382 forks source link

date parse error when running apt.deb command #725

Closed paolodina closed 2 years ago

paolodina commented 2 years ago

Describe the bug

Installing a .deb file from url fails.

$ pipx run pyinfra cubic ./software/chrome.py
    ...
    raise ParserError("Unknown string format: %s", timestr)
dateutil.parser._parser.ParserError: Unknown string format: gio 23 dic 2021, 11:02:05, CET

Error similar to #288.

To Reproduce

# ./software/chrome.py
from pyinfra.operations import apt

apt.deb(
    name='Install Chrome via deb',
    src='https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb',
    sudo=True,
)

Expected behavior

The command should end without errors.

Meta

pyinfra --support

System: Linux
  Platform: Linux-5.11.0-41-generic-x86_64-with-glibc2.29
  Release: 5.11.0-41-generic
  Machine: x86_64
pyinfra: v1.5
Executable: /home/paolo/.local/pipx/.cache/884e7a4f627c99d/bin/pyinfra
Python: 3.8.10 (CPython, GCC 9.3.0)

How was pyinfra installed (source/pip)?

I'm using pipx to run pyinfra (had no problems so far).

Include pyinfra-debug.log (if one was created)

Missing.

Output with -vv and --debug

$ pipx run pyinfra -vv --debug  cubic software/chrome.py 
    [pyinfra_cli.main] Checking potential directory: software
    [pyinfra_cli.main] Deploy directory remains as cwd
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...

--> Connecting to hosts...
    [pyinfra.api.connectors.ssh] Connecting to: cubic ({'allow_agent': True, 'look_for_keys': True, 'hostname': 'cubic', '_pyinfra_force_forward_agent': None, 'timeout': 10})
    [cubic] Connected
    [pyinfra.api.state] Activating host: cubic

--> Preparing operations...
    Loading: software/chrome.py
    [pyinfra.api.operation] Adding operation, {'Install Chrome via deb'}, opOrder=(3,), opHash=08743582456b52abe1182f5a5a3e12b457ba28b8
    [pyinfra.api.facts] Getting fact: file (path=/tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be) (ensure_hosts: (Host(cubic),))
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) sudo -H -n sh -c '! test -e /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be || ( stat -c '"'"'user=%U group=%G mode=%A atime=%X mtime=%Y ctime=%Z size=%s %N'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be 2> /dev/null || stat -f '"'"'user=%Su group=%Sg mode=%Sp atime=%a mtime=%m ctime=%c size=%z %N%SY'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be )'
[cubic] >>> sudo -H -n sh -c '! test -e /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be || ( stat -c '"'"'user=%U group=%G mode=%A atime=%X mtime=%Y ctime=%Z size=%s %N'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be 2> /dev/null || stat -f '"'"'user=%Su group=%Sg mode=%Sp atime=%a mtime=%m ctime=%c size=%z %N%SY'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be )'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 1
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) sh -c 'chmod +x pyinfra-sudo-askpass'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
[cubic] sudo password: 
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) env SUDO_ASKPASS=pyinfra-sudo-askpass *** sudo -H -A -k sh -c '! test -e /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be || ( stat -c '"'"'user=%U group=%G mode=%A atime=%X mtime=%Y ctime=%Z size=%s %N'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be 2> /dev/null || stat -f '"'"'user=%Su group=%Sg mode=%Sp atime=%a mtime=%m ctime=%c size=%z %N%SY'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be )'
[cubic] >>> env SUDO_ASKPASS=pyinfra-sudo-askpass *** sudo -H -A -k sh -c '! test -e /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be || ( stat -c '"'"'user=%U group=%G mode=%A atime=%X mtime=%Y ctime=%Z size=%s %N'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be 2> /dev/null || stat -f '"'"'user=%Su group=%Sg mode=%Sp atime=%a mtime=%m ctime=%c size=%z %N%SY'"'"' /tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be )'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    Loaded fact file (path=/tmp/pyinfra-f02e26d558402b781f9c7911421076ad992221be)
    [pyinfra.api.facts] Getting fact: date () (ensure_hosts: (Host(cubic),))
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) sudo -H -n sh -c 'LANG=C date'
[cubic] >>> sudo -H -n sh -c 'LANG=C date'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 1
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) env SUDO_ASKPASS=pyinfra-sudo-askpass *** sudo -H -A -k sh -c 'LANG=C date'
[cubic] >>> env SUDO_ASKPASS=pyinfra-sudo-askpass *** sudo -H -A -k sh -c 'LANG=C date'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [pyinfra.api.connectors.ssh] Running command on cubic: (pty=False) sh -c 'rm -f pyinfra-sudo-askpass'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
--> An unexpected exception occurred in: software/chrome.py:

  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra_cli/util.py", line 80, in exec_file
    exec(PYTHON_CODES[filename], data)
  File "software/chrome.py", line 3, in <module>
    apt.deb(
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/operation.py", line 365, in decorated_func
    commands = unroll_generators(func(*actual_args, **actual_kwargs))
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/util.py", line 194, in unroll_generators
    items.extend(unroll_generators(item))
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/util.py", line 192, in unroll_generators
    for item in generator:
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/operations/files.py", line 96, in download
    host_datetime = host.get_fact(Date).replace(tzinfo=None)
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/host.py", line 138, in get_fact
    return get_host_fact(self.state, self, name_or_cls, args=args, kwargs=kwargs)
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/facts.py", line 334, in get_host_fact
    fact_data = get_facts(state, name, args=args, kwargs=kwargs, ensure_hosts=(host,))
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/api/facts.py", line 290, in get_facts
    data = fact.process(stdout)
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/pyinfra/facts/server.py", line 129, in process
    return parse_date(output[0])
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/dateutil/parser/_parser.py", line 1368, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
  File "/home/paolo/.local/pipx/.cache/884e7a4f627c99d/lib/python3.8/site-packages/dateutil/parser/_parser.py", line 643, in parse
    raise ParserError("Unknown string format: %s", timestr)
dateutil.parser._parser.ParserError: Unknown string format: gio 23 dic 2021, 11:17:26, CET
Fizzadar commented 2 years ago

Hi @paolodina this may be a limitation of the python-dateutil library, which version do you have installed?

That said, I think pyinfra should handle this situation and show a warning rather than bailing on the exception.

paolodina commented 2 years ago

Hi Nick, I have the latest 2.8.2. I'm not sure about side effects, but as a workaround I specified a different locale via env:


apt.deb(
    ....
    env={'LC_TIME': "en_US.UTF-8"}
)
Fizzadar commented 2 years ago

I just released v1.6.2 with a change that always sets LC_TIME for the date fact which should fix this properly (https://github.com/Fizzadar/pyinfra/commit/6e94ab59886a7062afcb45467469c154731aead1).