pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.9k stars 381 forks source link

files.put() - incompatibility when `src` is `StringIO` and `dest` is a directory #1144

Open rsyring opened 3 months ago

rsyring commented 3 months ago

I was using file.put() with a directory destination and switched it to a file.template(). That resulted in the following traceback:

  File "/home/rsyring/.local/pipx/venvs/kilo/lib/python3.11/site-packages/pyinfra/operations/files.py", line 938, in put
    dest = unix_path_join(dest, os.path.basename(src))
                                ^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 142, in basename
TypeError: expected str, bytes or os.PathLike object, not StringIO

Once I figured out what was going on, the fix was to specify the dest as a file path, which makes sense.

Requests:

  1. A better error message indicating the root problem, i.e. "When using IO-like object for src a directory can not be used as the dest."
  2. Maybe not worth the effort, but when using files.template() and the dest is a directory, source the file name from the src file and add that to the dest before calling files.put(). Removing '.j2' or '.jinja' if present on the src filename.
simonhammes commented 4 days ago

An assert statement was added as part of #1082:

https://github.com/pyinfra-dev/pyinfra/blob/9ce7ac44ef6e9d5d8d8a926836e8d294cccbff29/pyinfra/operations/files.py#L854-L856

The following code now causes an AssertionError:

from pyinfra.operations import files

files.template(name='Template', src='template.j2', dest='/tmp')

However, the error message could (or should, imho) certainly be improved.


Maybe not worth the effort, but when using files.template() and the dest is a directory, source the file name from the src file and add that to the dest before calling files.put(). Removing '.j2' or '.jinja' if present on the src filename.

@Fizzadar What is your take on that? Makes sense or is that a bit too much "magic"?