Windows: Double carriage-returns in captured output

[robertschweizer]

Problem

Using example tasks.py:

from invoke import task

@task
def run_pyv(ctx):
    ctx.run("python -V")

I get the following output in CMD (wrapping in Python to see newline characters):

python -c "import subprocess; print(subprocess.run(['invoke', 'run-pyv'], capture_output=True).stdout)"
b'Python 3.8.10\r\r\n'

In most applications and terminals (e.g. CMD, Git bash etc.), the double carriage returns are no problem and are just ignored. I only faced issues in Gitlab's CI output: https://gitlab.com/gitlab-org/gitlab/-/issues/342686

Potential Solution

invoke currently uses os.read() to read the subprocess' output stream as bytes and then decodes it to str. This string contains \r\n line endings on Windows.

Then, invoke encodes the captured string (on Python 2 only) and prints it using sys.stdout.write(), which uses TextIOWrapper's newline handling. By default, this replaces any found \n with \r\n on Windows, ending up with \r\r\n.

To solve this, invoke should probably print the captured bytes before decoding, by using sys.stdout.buffer.write(), which does not do any newline handling and thus matches the os.read() used for reading.

[shoreadmin]

I've just stumbled across this problem as well. A workaround that seems to be working for me is to set the newline parameter on sys.stdout using the following:

import sys
sys.stdout.reconfigure(newline='')

[EugenSusurrus]

Looking forward to a fix for this as well. Have been using Invoke tasks in gitlab since longer time now because it's powerful and clean CLIs