1.0.0: pytest is failing in few units

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-python-frontmatter-1.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-python-frontmatter-1.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/python-frontmatter-1.0.0 collected 59 items tests/test_docs.py ... [ 5%] tests/test_files.py ............F......... [ 42%] tests/unit_test.py ..............F..FF...s....s.FFFsF [100%] ========================================================================================= FAILURES ========================================================================================== ___________________________________________________________________________________ test_file[filename1] ____________________________________________________________________________________ filename = PosixPath('/home/tkloczko/rpmbuild/BUILD/python-frontmatter-1.0.0/tests/toml/hello-toml.md') @pytest.mark.parametrize("filename", list(files())) def test_file(filename): result = Path(get_result_filename(filename)) if not result.exists(): pytest.fail(f"{result.name} does not exist") post = frontmatter.load(filename) result = json.loads(result.read_text()) > assert post.to_dict() == result E assert {'content': "...ldn't break."} == {'author': 'b...st': 'tester'} E Differing items: E {'content': '+++\nauthor = "bob"\nsomething = "else"\ntest = "tester"\n+++\n\nTitle\n=====\n\ntitle2\n------\n\nHello.\n\nJust need three dashes\n---\n\nAnd this shouldn\'t break.'} != {'content': "Title\n=====\n\ntitle2\n------\n\nHello.\n\nJust need three dashes\n---\n\nAnd this shouldn't break."} E Right contains 3 more items: E {'author': 'bob', 'something': 'else', 'test': 'tester'} E Use -v to get more diff tests/test_files.py:46: AssertionError ______________________________________________________________________________ HandlerTest.test_detect_format _______________________________________________________________________________ self = def test_detect_format(self): "detect format based on default handlers" for filename, Handler in self.TEST_FILES.items(): with codecs.open(filename, "r", "utf-8") as f: format = frontmatter.detect_format(f.read(), frontmatter.handlers) > self.assertIsInstance(format, Handler) E TypeError: isinstance() arg 2 must be a type or tuple of types tests/unit_test.py:190: TypeError ________________________________________________________________________________ HandlerTest.test_sanity_all ________________________________________________________________________________ self = def test_sanity_all(self): "Run sanity check on all handlers" for filename, Handler in self.TEST_FILES.items(): > self.sanity_check(filename, Handler) tests/unit_test.py:195: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/unit_test.py:175: in sanity_check self.assertIsInstance(post.handler, handler_type) E TypeError: isinstance() arg 2 must be a type or tuple of types ___________________________________________________________________________________ HandlerTest.test_toml ___________________________________________________________________________________ self = def test_toml(self): "load toml frontmatter" if toml is None: return post = frontmatter.load("tests/toml/hello-toml.md") metadata = {"author": "bob", "something": "else", "test": "tester"} for k, v in metadata.items(): > self.assertEqual(post[k], v) tests/unit_test.py:243: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , name = 'author' def __getitem__(self, name): "Get metadata key" > return self.metadata[name] E KeyError: 'author' frontmatter/__init__.py:275: KeyError ________________________________________________________________________________ TOMLHandlerTest.test_detect ________________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:385: TypeError _______________________________________________________________________________ TOMLHandlerTest.test_external _______________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:385: TypeError ____________________________________________________________________________ TOMLHandlerTest.test_split_content _____________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:385: TypeError ______________________________________________________________________________ TOMLHandlerTest.test_split_load ______________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:385: TypeError ================================================================================== short test summary info ================================================================================== SKIPPED [3] tests/unit_test.py:323: metadata can be reordered FAILED tests/test_files.py::test_file[filename1] - assert {'content': "...ldn't break."} == {'author': 'b...st': 'tester'} FAILED tests/unit_test.py::HandlerTest::test_detect_format - TypeError: isinstance() arg 2 must be a type or tuple of types FAILED tests/unit_test.py::HandlerTest::test_sanity_all - TypeError: isinstance() arg 2 must be a type or tuple of types FAILED tests/unit_test.py::HandlerTest::test_toml - KeyError: 'author' FAILED tests/unit_test.py::TOMLHandlerTest::test_detect - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_external - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_split_content - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_split_load - TypeError: 'NoneType' object is not callable ========================================================================== 8 failed, 48 passed, 3 skipped in 0.31s ========================================================================== ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.13 Babel 2.12.1 build 0.10.0 charset-normalizer 3.1.0 distro 1.8.0 docutils 0.19 exceptiongroup 1.0.0 gpg 1.19.0 idna 3.4 imagesize 1.4.1 importlib-metadata 6.5.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.2 libcomps 0.1.19 MarkupSafe 2.1.2 packaging 23.1 pluggy 1.0.0 Pygments 2.15.1 pyproject_hooks 1.0.0 pytest 7.3.1 python-dateutil 2.8.2 pytz 2023.2 PyYAML 6.0 requests 2.28.2 setuptools 67.6.1 six 1.16.0 snowballstemmer 2.2.0 Sphinx 6.1.3 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2.dev20230415 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1.dev20230415 sphinxcontrib-qthelp 1.0.3.dev20230415 sphinxcontrib-serializinghtml 1.1.5 tomli 2.0.1 urllib3 1.26.15 wheel 0.40.0 zipp 3.15.0 ```

[kloczek]

FYI I'm assuming that it is more likely test suite than tesed code issue so I've added in my build procedure those units to --deselect list.

[eyeseast]

This is fixed on main. Will cut a new release soon.

[kloczek]

Just tested 1.0.1 and pytest still fails 😞

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-frontmatter-1.0.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-frontmatter-1.0.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/python-frontmatter-1.0.1 collected 56 items tests/test_docs.py ... [ 5%] tests/test_files.py ............F......... [ 44%] tests/unit_test.py ..............F..FF........FFFF [100%] ========================================================================================= FAILURES ========================================================================================== ___________________________________________________________________________________ test_file[filename1] ____________________________________________________________________________________ filename = PosixPath('/home/tkloczko/rpmbuild/BUILD/python-frontmatter-1.0.1/tests/toml/hello-toml.md') @pytest.mark.parametrize("filename", list(files())) def test_file(filename): result = Path(get_result_filename(filename)) if not result.exists(): pytest.fail(f"{result.name} does not exist") post = frontmatter.load(filename) result = json.loads(result.read_text()) > assert post.to_dict() == result E assert {'content': "...ldn't break."} == {'author': 'b...st': 'tester'} E Differing items: E {'content': '+++\nauthor = "bob"\nsomething = "else"\ntest = "tester"\n+++\n\nTitle\n=====\n\ntitle2\n------\n\nHello.\n\nJust need three dashes\n---\n\nAnd this shouldn\'t break.'} != {'content': "Title\n=====\n\ntitle2\n------\n\nHello.\n\nJust need three dashes\n---\n\nAnd this shouldn't break."} E Right contains 3 more items: E {'author': 'bob', 'something': 'else', 'test': 'tester'} E Use -v to get more diff tests/test_files.py:46: AssertionError ______________________________________________________________________________ HandlerTest.test_detect_format _______________________________________________________________________________ self = def test_detect_format(self): "detect format based on default handlers" for filename, Handler in self.TEST_FILES.items(): with codecs.open(filename, "r", "utf-8") as f: format = frontmatter.detect_format(f.read(), frontmatter.handlers) > self.assertIsInstance(format, Handler) E TypeError: isinstance() arg 2 must be a type or tuple of types tests/unit_test.py:188: TypeError ________________________________________________________________________________ HandlerTest.test_sanity_all ________________________________________________________________________________ self = def test_sanity_all(self): "Run sanity check on all handlers" for filename, Handler in self.TEST_FILES.items(): > self.sanity_check(filename, Handler) tests/unit_test.py:193: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/unit_test.py:173: in sanity_check self.assertIsInstance(post.handler, handler_type) E TypeError: isinstance() arg 2 must be a type or tuple of types ___________________________________________________________________________________ HandlerTest.test_toml ___________________________________________________________________________________ self = def test_toml(self): "load toml frontmatter" if toml is None: return post = frontmatter.load("tests/toml/hello-toml.md") metadata = {"author": "bob", "something": "else", "test": "tester"} for k, v in metadata.items(): > self.assertEqual(post[k], v) tests/unit_test.py:241: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , name = 'author' def __getitem__(self, name): "Get metadata key" > return self.metadata[name] E KeyError: 'author' frontmatter/__init__.py:275: KeyError ________________________________________________________________________________ TOMLHandlerTest.test_detect ________________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:374: TypeError _______________________________________________________________________________ TOMLHandlerTest.test_external _______________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:374: TypeError ____________________________________________________________________________ TOMLHandlerTest.test_split_content _____________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:374: TypeError ______________________________________________________________________________ TOMLHandlerTest.test_split_load ______________________________________________________________________________ self = def setUp(self): > self.handler = TOMLHandler() E TypeError: 'NoneType' object is not callable tests/unit_test.py:374: TypeError ================================================================================== short test summary info ================================================================================== FAILED tests/test_files.py::test_file[filename1] - assert {'content': "...ldn't break."} == {'author': 'b...st': 'tester'} FAILED tests/unit_test.py::HandlerTest::test_detect_format - TypeError: isinstance() arg 2 must be a type or tuple of types FAILED tests/unit_test.py::HandlerTest::test_sanity_all - TypeError: isinstance() arg 2 must be a type or tuple of types FAILED tests/unit_test.py::HandlerTest::test_toml - KeyError: 'author' FAILED tests/unit_test.py::TOMLHandlerTest::test_detect - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_external - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_split_content - TypeError: 'NoneType' object is not callable FAILED tests/unit_test.py::TOMLHandlerTest::test_split_load - TypeError: 'NoneType' object is not callable =============================================================================== 8 failed, 48 passed in 0.30s ================================================================================```