linkchecker / linkchecker

check links in web documents or full websites
https://linkchecker.github.io/linkchecker/
GNU General Public License v2.0
870 stars 143 forks source link

Python 3.13: The URL with content type 'text/markdown' is not parseable (rather than application/octet-stream) #823

Closed hroncok closed 2 weeks ago

hroncok commented 3 months ago

Summary

When we package linkchecker 10.4.0 in Fedora and we build it with Python 3.13.0b1, we see this failure:

____________________________ TestFile.test_markdown ____________________________
[gw5] linux -- Python 3.13.0 /usr/bin/python3
tests/checker/test_file.py:90: in test_markdown
    self.file_test("file.markdown", confargs=confargs)
tests/checker/__init__.py:252: in file_test
    self.fail(msg)
E   AssertionError: tests/checker/data/file.markdown
E   --- expected
E   +++ result
E   @@ -2,7 +2,7 @@
E    cache key file:///builddir/build/BUILD/linkchecker-10.4.0-build/LinkChecker-10.4.0/tests/checker/data/file.markdown
E    real url file:///builddir/build/BUILD/linkchecker-10.4.0-build/LinkChecker-10.4.0/tests/checker/data/file.markdown
E    name tests/checker/data/file.markdown
E   -warning The URL with content type 'application/octet-stream' is not parseable.
E   +warning The URL with content type 'text/markdown' is not parseable.
E    valid
E    url http://url.example.com
E    cache key http://url.example.com
------------------------------ Captured log call -------------------------------
WARNING  linkcheck.check:log.py:67 Sitemap URL 'file:///builddir/build/BUILD/linkchecker-10.4.0-build/LinkChecker-10.4.0/tests/checker/data/file.markdown' does not start with http: or https:.

I suspect https://github.com/python/cpython/pull/17995 is the cause. This is being backported to Python 3.12 as well.

Steps to reproduce

  1. Run tests with Python 3.13 (3.13.0b1+) or Python 3.12 (from https://github.com/python/cpython/pull/118594)

Actual result

See above.

Expected result

Tests pass.

Environment

hroncok commented 3 months ago

I know how to fix this for the new Python, but I don't know how to make the test work with both old and new.

cjmayo commented 2 weeks ago

And now they backported it to.3.12.5!

I think the answer is just to include both, in a way that one can easily be removed when Python 3.12 is eventually dropped - Part of #828 (cffi is out for Python 3.13 now so we can claim support with rc1).