Testing output is dependent on terminal width

[Hritik14]

~After https://github.com/pallets/click/pull/1829~, while using

        runner = CliRunner()
        result = runner.invoke(...)

result.output contains the newline introduced by the wrappings due to auto detected terminal width. This is undesirable in test cases. As the docs says that these tools are made for testing only, IMO it would be better if the terminal width wrapping could be avoided in the output. For now, I'm using runner.invoke(terminal_width=1000, ...) in my test cases. Maybe having terminal_width=0 or a negative number could be treated an infinitely large terminal.

Environment:

• Python version: Python 3.9.5
• Click version: click==8.0.1

[davidism]

The linked PR does not include any changes that would result in the behavior you've described.

[davidism]

The test runner already forces the wrapping width and ignores terminal width.

[Hritik14]

@davidism Thank you for the quick response. Here's the test run I tried to reproduce the issue.

test_cli.py

from click.testing import CliRunner
from cli import cli

def test_help():
    runner = CliRunner()
    result = runner.invoke(cli, ['--help'])

    print(result.output)

    result_large_width = runner.invoke(cli, ['--help'], terminal_width=1000)
    print(result_large_width.output)

    assert result.output == result_large_width.output

if __name__ == "__main__":
    test_help()

cli.py

import click

@click.command()
@click.help_option('-h', '--help')
def cli():
    """
    Identify the changes that need to be made to the 'old'
    scan file (-o or --old) in order to generate the 'new' scan file (-n or
    --new).  Write the results to a .json file (-j or --json-file) at a
    user-designated location.  If no file option is selected, print the JSON
    results to the console.
    """
    ...

if __name__ == '__main__':
    cli()

Sample test:

/tmp/test_project
venv ❯ python test_cli.py
Usage: cli [OPTIONS]

  Identify the changes that need to be made to the 'old' scan file (-o or --old)
  in order to generate the 'new' scan file (-n or --new).  Write the results to
  a .json file (-j or --json-file) at a user-designated location.  If no file
  option is selected, print the JSON results to the console.

Options:
  -h, --help  Show this message and exit.

Usage: cli [OPTIONS]

  Identify the changes that need to be made to the 'old' scan file (-o or --old) in order to generate the 'new' scan file (-n or --new).  Write the results to a .json file (-j or --json-file) at a user-designated location.  If no file option is selected, print the JSON results to the console.

Options:
  -h, --help  Show this message and exit.

Traceback (most recent call last):
  File "/tmp/test_project/test_cli.py", line 16, in <module>
    test_help()
  File "/tmp/test_project/test_cli.py", line 13, in test_help
    assert result.output == result_large_width.output
AssertionError

[davidism]

If you can do git bisect to identify the commit that introduced the behavior, that would be helpful, as the pr you linked is not correct.

[Hritik14]

The linked PR does not include any changes that would result in the behavior you've described.

You're right. The linked PR doesn't include those changes. It just so happened that the linked PR changes the output that broke my test cases by introducing different formatting.

At ddcabdd4f22f438c0c963150b930d0d09b04dea7

{"Usage: cli [OPTIONS]\n\n  Identify the changes that need to be made to the 'old' scan file (-o or --old)\n  in order to generate the 'new' scan file (-n or --new).  Write the results to\n  a .json file (-j or --json-file) at a user-designated location.  If no file\n  option is selected, print the JSON results to the console.\n\nOptions:\n  -h, --help  Show this message and exit.\n"}

At 8d49e14^1

{"Usage: cli [OPTIONS]\n\n  Identify the changes that need to be made to the 'old' scan file (-o or\n  --old) in order to generate the 'new' scan file (-n or --new).  Write the\n  results to a .json file (-j or --json-file) at a user-designated location.\n  If no file option is selected, print the JSON results to the console.\n\nOptions:\n  -h, --help  Show this message and exit.\n"}

[Hritik14]

If you can do git bisect to identify the commit that introduced the behavior, that would be helpful

Sorry to say, I tried the very first commit that brought testing into the API (c265355bcd50a0848cb973e8e83b3e21ee60a775). This looks like something that was always there.

[MVrachev]

I think I got the same error on different terminals (MAC built-in terminal, Iterm2 and VScode terminals). I was able to fail all our project test cases that are asserting strings against ClickRunner command output.