search

pypa / readme_renderer

Safely render long_description/README files in Warehouse
Apache License 2.0
158 stars 88 forks source link

37.3: pytest fails in in 6 units #275

Closed wuch-g2v closed 1 year ago

wuch-g2v commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-readme_renderer-37.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-readme_renderer-37.3-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.16, pytest-7.3.0, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3 collected 48 items tests/test_clean.py . [ 2%] tests/test_cli.py FF..FF...F.F... [ 33%] tests/test_markdown.py s. [ 37%] tests/test_noextra.py .. [ 41%] tests/test_rst.py ........................... [ 97%] tests/test_txt.py . [100%] ========================================================================================= FAILURES ========================================================================================== _____________________________________________________________________ test_cli_input_file[test_CommonMark_001.md-False] _____________________________________________________________________ input_file = PosixPath('tests/fixtures/test_CommonMark_001.md'), output_file = False @pytest.mark.parametrize("output_file", [False, True]) def test_cli_input_file(input_file, output_file): with mock.patch("builtins.print") as print_: if output_file: with tempfile.TemporaryDirectory() as tmpdir: output = pathlib.Path(tmpdir) / "output.html" main(["-o", str(output), str(input_file)]) with output.open() as fp: result = fp.read() else: > main([str(input_file)]) tests/test_cli.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['tests/fixtures/test_CommonMark_001.md'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit _____________________________________________________________________ test_cli_input_file[test_CommonMark_001.md-True] ______________________________________________________________________ input_file = PosixPath('tests/fixtures/test_CommonMark_001.md'), output_file = True @pytest.mark.parametrize("output_file", [False, True]) def test_cli_input_file(input_file, output_file): with mock.patch("builtins.print") as print_: if output_file: with tempfile.TemporaryDirectory() as tmpdir: output = pathlib.Path(tmpdir) / "output.html" > main(["-o", str(output), str(input_file)]) tests/test_cli.py:20: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['-o', '/tmp/tmp33u9of8i/output.html', 'tests/fixtures/test_CommonMark_001.md'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit ________________________________________________________________________ test_cli_input_file[test_GFM_001.md-False] _________________________________________________________________________ input_file = PosixPath('tests/fixtures/test_GFM_001.md'), output_file = False @pytest.mark.parametrize("output_file", [False, True]) def test_cli_input_file(input_file, output_file): with mock.patch("builtins.print") as print_: if output_file: with tempfile.TemporaryDirectory() as tmpdir: output = pathlib.Path(tmpdir) / "output.html" main(["-o", str(output), str(input_file)]) with output.open() as fp: result = fp.read() else: > main([str(input_file)]) tests/test_cli.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['tests/fixtures/test_GFM_001.md'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit _________________________________________________________________________ test_cli_input_file[test_GFM_001.md-True] _________________________________________________________________________ input_file = PosixPath('tests/fixtures/test_GFM_001.md'), output_file = True @pytest.mark.parametrize("output_file", [False, True]) def test_cli_input_file(input_file, output_file): with mock.patch("builtins.print") as print_: if output_file: with tempfile.TemporaryDirectory() as tmpdir: output = pathlib.Path(tmpdir) / "output.html" > main(["-o", str(output), str(input_file)]) tests/test_cli.py:20: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['-o', '/tmp/tmp6if1era7/output.html', 'tests/fixtures/test_GFM_001.md'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit _____________________________________________________________________ test_cli_explicit_format[test_CommonMark_001.md] ______________________________________________________________________ input_file = PosixPath('tests/fixtures/test_CommonMark_001.md') def test_cli_explicit_format(input_file): fmt = input_file.suffix.lstrip(".") with input_file.open() as fp, \ mock.patch("pathlib.Path.open", return_value=fp), \ mock.patch("builtins.print") as print_: > main(["-f", fmt, "no-file.invalid"]) tests/test_cli.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['-f', 'md', 'no-file.invalid'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit _________________________________________________________________________ test_cli_explicit_format[test_GFM_001.md] _________________________________________________________________________ input_file = PosixPath('tests/fixtures/test_GFM_001.md') def test_cli_explicit_format(input_file): fmt = input_file.suffix.lstrip(".") with input_file.open() as fp, \ mock.patch("pathlib.Path.open", return_value=fp), \ mock.patch("builtins.print") as print_: > main(["-f", fmt, "no-file.invalid"]) tests/test_cli.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cli_args = ['-f', 'md', 'no-file.invalid'] def main(cli_args: Optional[List[str]] = None) -> None: parser = argparse.ArgumentParser( description="Renders a .md, .rst, or .txt README to HTML", ) parser.add_argument("-p", "--package", help="Get README from package metadata", action="store_true") parser.add_argument("-f", "--format", choices=["md", "rst", "txt"], help="README format (inferred from input file name or package)") parser.add_argument('input', help="Input README file or package name") parser.add_argument('-o', '--output', help="Output file (default: stdout)", type=argparse.FileType('w'), default='-') args = parser.parse_args(cli_args) content_format = args.format if args.package: distribution = get_distribution(args.input) pkg_info = distribution.get_metadata(distribution.PKG_INFO) message = email.message_from_string(pkg_info) source = message.get_payload() # Infer the format of the description from package metadata. if not content_format: content_type = message.get("Description-Content-Type", "text/x-rst") if content_type == "text/x-rst": content_format = "rst" elif content_type == "text/markdown": content_format = "md" elif content_type == "text/plain": content_format = "txt" else: raise ValueError(f"invalid content type {content_type} for package " "`long_description`") else: filename = pathlib.Path(args.input) content_format = content_format or filename.suffix.lstrip(".") with filename.open() as fp: source = fp.read() if content_format == "md": rendered = render_md(source, stream=sys.stderr) elif content_format == "rst": rendered = render_rst(source, stream=sys.stderr) elif content_format == "txt": rendered = render_txt(source, stream=sys.stderr) else: raise ValueError(f"invalid README format: {content_format} (expected `md`, " "`rst`, or `txt`)") if rendered is None: > sys.exit(1) E SystemExit: 1 readme_renderer/__main__.py:60: SystemExit ===================================================================================== warnings summary ====================================================================================== readme_renderer/markdown.py:44 /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/readme_renderer/markdown.py:44: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering. warnings.warn(_EXTRA_WARNING) ../../../../../usr/lib/python3.8/site-packages/pkg_resources/__init__.py:121 /usr/lib/python3.8/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning) tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-False] tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-True] tests/test_cli.py::test_cli_input_file[test_GFM_001.md-False] tests/test_cli.py::test_cli_input_file[test_GFM_001.md-True] tests/test_cli.py::test_cli_explicit_format[test_CommonMark_001.md] tests/test_cli.py::test_cli_explicit_format[test_GFM_001.md] tests/test_markdown.py::test_missing_variant /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/readme_renderer/markdown.py:60: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering. warnings.warn(_EXTRA_WARNING) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_markdown.py:8: got empty parameter set ('md_filename', 'html_filename', 'variant'), function test_md_fixtures at /home/tkloczko/rpmbuild/BUILD/readme_renderer-37.3/tests/test_markdown.py:7 FAILED tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-False] - SystemExit: 1 FAILED tests/test_cli.py::test_cli_input_file[test_CommonMark_001.md-True] - SystemExit: 1 FAILED tests/test_cli.py::test_cli_input_file[test_GFM_001.md-False] - SystemExit: 1 FAILED tests/test_cli.py::test_cli_input_file[test_GFM_001.md-True] - SystemExit: 1 FAILED tests/test_cli.py::test_cli_explicit_format[test_CommonMark_001.md] - SystemExit: 1 FAILED tests/test_cli.py::test_cli_explicit_format[test_GFM_001.md] - SystemExit: 1 ==================================================================== 6 failed, 41 passed, 1 skipped, 9 warnings in 0.83s ==================================================================== ```

Here is list of installed modules in build env

```console Package Version --------------- ------- bleach 6.0.0 build 0.10.0 distro 1.8.0 docutils 0.19 exceptiongroup 1.0.0 gpg 1.19.0 html5lib 1.1 iniconfig 2.0.0 installer 0.7.0 libcomps 0.1.19 packaging 23.0 pluggy 1.0.0 Pygments 2.15.0 pyproject_hooks 1.0.0 pytest 7.3.0 python-dateutil 2.8.2 setuptools 67.6.1 six 1.16.0 tomli 2.0.1 webencodings 0.5.1 wheel 0.40.0 ```
miketheman commented 1 year ago

Thanks for opening this.

@wuch-g2v looks like you're pulling unreleased code from github, is that correct?

Looks like the recent changes in #271 did not account for the noextra use case where the markdown extensions are not installed, and only for running the test case

Can also be reproduced via tox -e noextra.

@tillahoffmann would you want to take a pass at fixing this?