3.1.0: pytest is failing

Just normal build, install and test cycle used on building package from non-root account:

"setup.py build"
"setup.py install --root </install/prefix>"
"pytest with PYTHONPATH pointing to setearch and sitelib inside </install/prefix>
PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.1.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.1.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
PYTHONDONTWRITEBYTECODE=1
/usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.1.0 plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, cases-3.6.1, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, Faker-8.8.2, datadir-1.3.1, regressions-2.2.0 collected 13 items

. . [ 7%] tests/test_build.py .....ss..E.. [100%]

================================================================================== ERRORS ================================================================================== _ ERROR at setup of test_rinohtypepdf

self = <sphinx.registry.SphinxComponentRegistry object at 0x7fce1f157640> app = <[AttributeError("'NoneType' object has no attribute 'name'") raised in repr()] SphinxTestApp object at 0x7fce1f1630a0>, extname = 'rinoh.frontend.sphinx'

def load_extension(self, app: "Sphinx", extname: str) -> None:
    """Load a Sphinx extension."""
    if extname in app.extensions:  # already loaded
        return
    if extname in EXTENSION_BLACKLIST:
        logger.warning(__('the extension %r was already merged with Sphinx since '
                          'version %s; this extension is ignored.'),
                       extname, EXTENSION_BLACKLIST[extname])
        return

    # update loading context
    prefix = __('while setting up extension %s:') % extname
    with prefixed_warnings(prefix):
        try:

          mod = import_module(extname)

/usr/lib/python3.8/site-packages/sphinx/registry.py:420:

name = 'rinoh.frontend.sphinx', package = None

def import_module(name, package=None):
    """Import a module.

    The 'package' argument is required when performing a relative import. It
    specifies the package to use as the anchor point from which to resolve the
    relative import to an absolute import.

    """
    level = 0
    if name.startswith('.'):
        if not package:
            msg = ("the 'package' argument is required to perform a relative "
                   "import for {!r}")
            raise TypeError(msg.format(name))
        for character in name:
            if character != '.':
                break
            level += 1

  return _bootstrap._gcd_import(name[level:], package, level)

/usr/lib64/python3.8/importlib/init.py:127:

name = 'rinoh.frontend.sphinx', package = None, level = 0

???

:1014: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh.frontend.sphinx', import_ = > ??? :991: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh.frontend.sphinx', import_ = > ??? :961: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ f = , args = ('rinoh.frontend',), kwds = {} > ??? :219: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh.frontend', package = None, level = 0 > ??? :1014: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh.frontend', import_ = > ??? :991: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh.frontend', import_ = > ??? :961: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ f = , args = ('rinoh',), kwds = {} > ??? :219: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh', package = None, level = 0 > ??? :1014: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh', import_ = > ??? :991: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ name = 'rinoh', import_ = > ??? E ModuleNotFoundError: No module named 'rinoh' :973: ModuleNotFoundError The above exception was the direct cause of the following exception: test_params = {'shared_result': None}, app_params = app_params(args=['rinoh'], kwargs={'srcdir': path('/tmp/pytest-of-tkloczko/pytest-227/rinohtype-pdf')}) make_app = .make at 0x7fce1f23b4c0>, shared_result = @pytest.fixture(scope='function') def app(test_params: Dict, app_params: Tuple[Dict, Dict], make_app: Callable, shared_result: SharedResult) -> Generator[SphinxTestApp, None, None]: """ provides sphinx.application.Sphinx object """ args, kwargs = app_params > app_ = make_app(*args, **kwargs) /usr/lib/python3.8/site-packages/sphinx/testing/fixtures.py:147: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.8/site-packages/sphinx/testing/fixtures.py:193: in make app_: Any = SphinxTestApp(*args, **kwargs) /usr/lib/python3.8/site-packages/sphinx/testing/util.py:130: in __init__ super().__init__(srcdir, confdir, outdir, doctreedir, /usr/lib/python3.8/site-packages/sphinx/application.py:243: in __init__ self.setup_extension(extension) /usr/lib/python3.8/site-packages/sphinx/application.py:400: in setup_extension self.registry.load_extension(self, extname) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = app = <[AttributeError("'NoneType' object has no attribute 'name'") raised in repr()] SphinxTestApp object at 0x7fce1f1630a0>, extname = 'rinoh.frontend.sphinx' def load_extension(self, app: "Sphinx", extname: str) -> None: """Load a Sphinx extension.""" if extname in app.extensions: # already loaded return if extname in EXTENSION_BLACKLIST: logger.warning(__('the extension %r was already merged with Sphinx since ' 'version %s; this extension is ignored.'), extname, EXTENSION_BLACKLIST[extname]) return # update loading context prefix = __('while setting up extension %s:') % extname with prefixed_warnings(prefix): try: mod = import_module(extname) except ImportError as err: logger.verbose(__('Original exception:\n') + traceback.format_exc()) > raise ExtensionError(__('Could not import extension %s') % extname, err) from err E sphinx.errors.ExtensionError: Could not import extension rinoh.frontend.sphinx (exception: No module named 'rinoh') /usr/lib/python3.8/site-packages/sphinx/registry.py:423: ExtensionError ============================================================================= warnings summary ============================================================================= tests/test_build.py:54 /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.1.0/tests/test_build.py:54: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html @pytest.mark.noautobuild -- Docs: https://docs.pytest.org/en/stable/warnings.html ========================================================================= short test summary info ========================================================================== SKIPPED [1] tests/test_build.py:28: Test uses Sphinx 3 code blocks SKIPPED [1] tests/test_build.py:36: Test uses Sphinx 4 code blocks ERROR tests/test_build.py::test_rinohtype_pdf - sphinx.errors.ExtensionError: Could not import extension rinoh.frontend.sphinx (exception: No module named 'rinoh') ============================================================ 10 passed, 2 skipped, 1 warning, 1 error in 9.16s =============================================================``` I don't see on pypi `rinoh` module.

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

Ah mate, this is the 4th repo I’ve seen you open a similar issue 😬. rhinotype is clearly in the testing extra, that I assume you have not used. I feel you are expecting tests to work in a different way (in a bespoke environment with many other packages), rather than the way they are designed to work (in a very specific testing environment)

If you'd like to give this a try, I'd recommend using a virtual environment and installing extra dependencies as suggested using pip install sphinx-tabs[testing]

Just found that there are som new errors. In mean time I've upgraded sphins to 4.1.2.

___________________________________________________________ ERROR at setup of test_conditional_assets[no_tabs2] ____________________________________________________________

app = <SphinxTestApp buildername='html'>, status = <_io.StringIO object at 0x7f905a323f70>, warning = <_io.StringIO object at 0x7f905a323dc0>
check_build_success = <function check_build_success.<locals>.check at 0x7f905a21e040>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f905a21e160>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f905a21e280>
request = <SubRequest 'auto_build_and_check' for <Function test_conditional_assets[no_tabs2]>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

app = <SphinxTestApp buildername='html'>, buildername = 'html', filename = 'index.html', encoding = 'utf-8'

    def read(app, buildername="html", filename="index.html", encoding="utf-8"):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")

            # Remove output from ``pygments``, so that test only compares HTML of surrounding tabs
            for div in soup.find_all("div", {"class": "highlight"}):
                div.decompose()

            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()

        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs2_.html
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs2_.obtained.html
E       HTML DIFF: /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs2_.obtained.diff.html
E       ---
E       +++
E       @@ -15,7 +15,7 @@
E             </li>
E            </ul>
E           </div>
E       -   <div class="section" id="fruits">
E       +   <section id="fruits">
E            <h1>
E             Fruits
E             <a class="headerlink" href="#fruits" title="Permalink to this headline">
E       @@ -50,8 +50,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="luminaries">
E       +   </section>
E       +   <section id="luminaries">
E            <h1>
E             Luminaries
E             <a class="headerlink" href="#luminaries" title="Permalink to this headline">
E       @@ -78,8 +78,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="code-tabs">
E       +   </section>
E       +   <section id="code-tabs">
E            <h1>
E             Code Tabs
E             <a class="headerlink" href="#code-tabs" title="Permalink to this headline">
E       @@ -178,8 +178,8 @@
E              </div>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="group-tabs">
E       +   </section>
E       +   <section id="group-tabs">
E            <h1>
E             Group Tabs
E             <a class="headerlink" href="#group-tabs" title="Permalink to this headline">
E       @@ -242,7 +242,7 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       +   </section>
E          </div>
E         </div>
E        </div>

tests/conftest.py:93: AssertionError
___________________________________________________________________ ERROR at setup of test_nested_markup ___________________________________________________________________

app = <SphinxTestApp buildername='html'>, status = <_io.StringIO object at 0x7f905a00da60>, warning = <_io.StringIO object at 0x7f905a00daf0>
check_build_success = <function check_build_success.<locals>.check at 0x7f905a02b8b0>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f905a02b9d0>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f905a02baf0>
request = <SubRequest 'auto_build_and_check' for <Function test_nested_markup>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

app = <SphinxTestApp buildername='html'>, buildername = 'html', filename = 'index.html', encoding = 'utf-8'

    def read(app, buildername="html", filename="index.html", encoding="utf-8"):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")

            # Remove output from ``pygments``, so that test only compares HTML of surrounding tabs
            for div in soup.find_all("div", {"class": "highlight"}):
                div.decompose()

            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()

        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-tkloczko/pytest-117/test_nested_markup0/test_build/test_nested_markup.html
E       /tmp/pytest-of-tkloczko/pytest-117/test_nested_markup0/test_build/test_nested_markup.obtained.html
E       HTML DIFF: /tmp/pytest-of-tkloczko/pytest-117/test_nested_markup0/test_build/test_nested_markup.obtained.diff.html
E       ---
E       +++
E       @@ -1,7 +1,7 @@
E        <div class="documentwrapper">
E         <div class="bodywrapper">
E          <div class="body" role="main">
E       -   <div class="section" id="markup-in-tab-titles">
E       +   <section id="markup-in-tab-titles">
E            <h1>
E             Markup in Tab Titles
E             <a class="headerlink" href="#markup-in-tab-titles" title="Permalink to this headline">
E       @@ -206,7 +206,7 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       +   </section>
E          </div>
E         </div>
E        </div>

tests/conftest.py:93: AssertionError
___________________________________________________________ ERROR at setup of test_conditional_assets[no_tabs1] ____________________________________________________________

app = <SphinxTestApp buildername='html'>, status = <_io.StringIO object at 0x7f9059cd7310>, warning = <_io.StringIO object at 0x7f9059cd7040>
check_build_success = <function check_build_success.<locals>.check at 0x7f9059f29310>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f9059f291f0>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f9059f290d0>
request = <SubRequest 'auto_build_and_check' for <Function test_conditional_assets[no_tabs1]>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

app = <SphinxTestApp buildername='html'>, buildername = 'html', filename = 'index.html', encoding = 'utf-8'

    def read(app, buildername="html", filename="index.html", encoding="utf-8"):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")

            # Remove output from ``pygments``, so that test only compares HTML of surrounding tabs
            for div in soup.find_all("div", {"class": "highlight"}):
                div.decompose()

            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()

        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs1_.html
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs1_.obtained.html
E       HTML DIFF: /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs1_.obtained.diff.html
E       ---
E       +++
E       @@ -15,7 +15,7 @@
E             </li>
E            </ul>
E           </div>
E       -   <div class="section" id="fruits">
E       +   <section id="fruits">
E            <h1>
E             Fruits
E             <a class="headerlink" href="#fruits" title="Permalink to this headline">
E       @@ -50,8 +50,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="luminaries">
E       +   </section>
E       +   <section id="luminaries">
E            <h1>
E             Luminaries
E             <a class="headerlink" href="#luminaries" title="Permalink to this headline">
E       @@ -78,8 +78,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="code-tabs">
E       +   </section>
E       +   <section id="code-tabs">
E            <h1>
E             Code Tabs
E             <a class="headerlink" href="#code-tabs" title="Permalink to this headline">
E       @@ -178,8 +178,8 @@
E              </div>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="group-tabs">
E       +   </section>
E       +   <section id="group-tabs">
E            <h1>
E             Group Tabs
E             <a class="headerlink" href="#group-tabs" title="Permalink to this headline">
E       @@ -242,7 +242,7 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       +   </section>
E          </div>
E         </div>
E        </div>

tests/conftest.py:93: AssertionError
_____________________________________________________________ ERROR at setup of test_conditional_assets[index] _____________________________________________________________

app = <SphinxTestApp buildername='html'>, status = <_io.StringIO object at 0x7f9059fa93a0>, warning = <_io.StringIO object at 0x7f9059fa94c0>
check_build_success = <function check_build_success.<locals>.check at 0x7f9059fbac10>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f9059ddbe50>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f9059ddbf70>
request = <SubRequest 'auto_build_and_check' for <Function test_conditional_assets[index]>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

app = <SphinxTestApp buildername='html'>, buildername = 'html', filename = 'index.html', encoding = 'utf-8'

    def read(app, buildername="html", filename="index.html", encoding="utf-8"):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")

            # Remove output from ``pygments``, so that test only compares HTML of surrounding tabs
            for div in soup.find_all("div", {"class": "highlight"}):
                div.decompose()

            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()

        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.html
E       /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.obtained.html
E       HTML DIFF: /tmp/pytest-of-tkloczko/pytest-117/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.obtained.diff.html
E       ---
E       +++
E       @@ -15,7 +15,7 @@
E             </li>
E            </ul>
E           </div>
E       -   <div class="section" id="fruits">
E       +   <section id="fruits">
E            <h1>
E             Fruits
E             <a class="headerlink" href="#fruits" title="Permalink to this headline">
E       @@ -50,8 +50,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="luminaries">
E       +   </section>
E       +   <section id="luminaries">
E            <h1>
E             Luminaries
E             <a class="headerlink" href="#luminaries" title="Permalink to this headline">
E       @@ -78,8 +78,8 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="code-tabs">
E       +   </section>
E       +   <section id="code-tabs">
E            <h1>
E             Code Tabs
E             <a class="headerlink" href="#code-tabs" title="Permalink to this headline">
E       @@ -178,8 +178,8 @@
E              </div>
E             </div>
E            </div>
E       -   </div>
E       -   <div class="section" id="group-tabs">
E       +   </section>
E       +   <section id="group-tabs">
E            <h1>
E             Group Tabs
E             <a class="headerlink" href="#group-tabs" title="Permalink to this headline">
E       @@ -242,7 +242,7 @@
E              </p>
E             </div>
E            </div>
E       -   </div>
E       +   </section>
E          </div>
E         </div>
E        </div>

tests/conftest.py:93: AssertionError
============================================================================= warnings summary =============================================================================
tests/test_build.py:28
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.2.0/tests/test_build.py:28: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.noautobuild

tests/test_build.py:83
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.2.0/tests/test_build.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.noautobuild

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_build.py:57: Test uses Sphinx 3 code blocks
ERROR tests/test_build.py::test_conditional_assets[no_tabs2] - AssertionError: FILES DIFFER:
ERROR tests/test_build.py::test_nested_markup - AssertionError: FILES DIFFER:
ERROR tests/test_build.py::test_conditional_assets[no_tabs1] - AssertionError: FILES DIFFER:
ERROR tests/test_build.py::test_conditional_assets[index] - AssertionError: FILES DIFFER:

I feel you are expecting tests to work in a different way

Wrong feeling. I have no any expectiations .. I'm just humble messanger. Only this and nothing more :)

If you'd like to give this a try, I'd recommend using a virtual environment and installing extra dependencies as suggested using pip install sphinx-tabs[testing]

On that list is sphinx_testing which is no longer maintained and AFAIK it should not be used with sphinx 4.x. In other words looks like test suite is no redy for sphinx 4.x.

Other dependency listed in sphinx-tabs[testing] is "pytest>=3.6,<4". Latest pytest it is 6.2.5.

Other dependency listed in sphinx-tabs[testing] is "pytest>=3.6,<4". Latest pytest it is 6.2.5.

Yeah after deselecting failing units it is possible to see some pytest warnings

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.2.0-2.1.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.2.0-2.1.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect 'tests/test_build.py::test_conditional_assets_html_assets_policy[index]' --deselect 'tests/test_build.py::test_conditional_assets_html_assets_policy[no_tabs1]' --deselect 'tests/test_build.py::test_conditional_assets_html_assets_policy[no_tabs2]' --deselect 'tests/test_build.py::test_conditional_assets[index]' --deselect 'tests/test_build.py::test_conditional_assets[no_tabs1]' --deselect 'tests/test_build.py::test_conditional_assets[no_tabs2]' --deselect tests/test_build.py::test_nested_markup --deselect tests/test_build.py::test_rinohtype_pdf
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.2.0
plugins: datadir-1.3.1, cov-3.0.0, regressions-2.2.0
collected 15 items / 8 deselected / 7 selected

tests/test_build.py ..s....                                                                                                                                          [100%]

============================================================================= warnings summary =============================================================================
tests/test_build.py:28
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.2.0/tests/test_build.py:28: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.noautobuild

tests/test_build.py:83
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.2.0/tests/test_build.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.noautobuild

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_build.py:57: Test uses Sphinx 3 code blocks
========================================================== 6 passed, 1 skipped, 8 deselected, 2 warnings in 2.21s ==========================================================

Wrong feeling. I have no any expectiations .. I'm just humble messanger. Only this and nothing more :)

You have a different version of pytest installed to the one listed in our dependencies, and are reporting issues that we don't see in our continuous integration tests. This issue is closed, so we wouldn't like to discuss this point any further.

Your sphinx_testing comment is a valid point, we should look to replace this with a supported equivalent. But for now it has worked well with out tests when they are run in a correct environment.

You have a different version of pytest installed to the one listed in our dependencies, and are reporting issues that we don't see in our continuous integration tests. This issue is closed, so we wouldn't like to discuss this point any further.

Please don't get me wrong .. last pytest 3.x has been released +4 years ago. I've reported that issue because all other modules (and most of the pytest extensions) around alredy moved to pytast 6.x and alreary in testing stage is pytest 7.x.

Just tested 3.3.1. One units emmits error

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.3.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-tabs-3.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.3.1
plugins: datadir-1.3.1, regressions-2.3.1, cov-3.0.0
collected 15 items

tests/test_build.py ........s...E..                                                                                                                                  [100%]

================================================================================== ERRORS ==================================================================================
___________________________________________________________________ ERROR at setup of test_rinohtype_pdf ___________________________________________________________________

self = <sphinx.registry.SphinxComponentRegistry object at 0x7f6bee651cd0>
app = <[AttributeError("'NoneType' object has no attribute 'name'") raised in repr()] SphinxTestApp object at 0x7f6bee651610>, extname = 'rinoh.frontend.sphinx'

    def load_extension(self, app: "Sphinx", extname: str) -> None:
        """Load a Sphinx extension."""
        if extname in app.extensions:  # already loaded
            return
        if extname in EXTENSION_BLACKLIST:
            logger.warning(__('the extension %r was already merged with Sphinx since '
                              'version %s; this extension is ignored.'),
                           extname, EXTENSION_BLACKLIST[extname])
            return

        # update loading context
        prefix = __('while setting up extension %s:') % extname
        with prefixed_warnings(prefix):
            try:
>               mod = import_module(extname)

/usr/lib/python3.8/site-packages/sphinx/registry.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend.sphinx', package = None

    def import_module(name, package=None):
        """Import a module.

        The 'package' argument is required when performing a relative import. It
        specifies the package to use as the anchor point from which to resolve the
        relative import to an absolute import.

        """
        level = 0
        if name.startswith('.'):
            if not package:
                msg = ("the 'package' argument is required to perform a relative "
                       "import for {!r}")
                raise TypeError(msg.format(name))
            for character in name:
                if character != '.':
                    break
                level += 1
>       return _bootstrap._gcd_import(name[level:], package, level)

/usr/lib64/python3.8/importlib/__init__.py:127:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend.sphinx', package = None, level = 0

>   ???

<frozen importlib._bootstrap>:1014:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend.sphinx', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???

<frozen importlib._bootstrap>:991:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend.sphinx', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???

<frozen importlib._bootstrap>:961:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

f = <function _gcd_import at 0x7f6bf14704c0>, args = ('rinoh.frontend',), kwds = {}

>   ???

<frozen importlib._bootstrap>:219:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend', package = None, level = 0

>   ???

<frozen importlib._bootstrap>:1014:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???

<frozen importlib._bootstrap>:991:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh.frontend', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???

<frozen importlib._bootstrap>:961:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

f = <function _gcd_import at 0x7f6bf14704c0>, args = ('rinoh',), kwds = {}

>   ???

<frozen importlib._bootstrap>:219:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh', package = None, level = 0

>   ???

<frozen importlib._bootstrap>:1014:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???

<frozen importlib._bootstrap>:991:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'rinoh', import_ = <function _gcd_import at 0x7f6bf14704c0>

>   ???
E   ModuleNotFoundError: No module named 'rinoh'

<frozen importlib._bootstrap>:973: ModuleNotFoundError

The above exception was the direct cause of the following exception:

test_params = {'shared_result': None}, app_params = app_params(args=['rinoh'], kwargs={'srcdir': path('/tmp/pytest-of-tkloczko/pytest-74/rinohtype-pdf')})
make_app = <function make_app.<locals>.make at 0x7f6beeeae940>, shared_result = <sphinx.testing.fixtures.SharedResult object at 0x7f6bee030c70>

    @pytest.fixture(scope='function')
    def app(test_params: Dict, app_params: Tuple[Dict, Dict], make_app: Callable,
            shared_result: SharedResult) -> Generator[SphinxTestApp, None, None]:
        """
        Provides the 'sphinx.application.Sphinx' object
        """
        args, kwargs = app_params
>       app_ = make_app(*args, **kwargs)

/usr/lib/python3.8/site-packages/sphinx/testing/fixtures.py:139:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/sphinx/testing/fixtures.py:185: in make
    app_: Any = SphinxTestApp(*args, **kwargs)
/usr/lib/python3.8/site-packages/sphinx/testing/util.py:125: in __init__
    super().__init__(srcdir, confdir, outdir, doctreedir,
/usr/lib/python3.8/site-packages/sphinx/application.py:223: in __init__
    self.setup_extension(extension)
/usr/lib/python3.8/site-packages/sphinx/application.py:380: in setup_extension
    self.registry.load_extension(self, extname)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sphinx.registry.SphinxComponentRegistry object at 0x7f6bee651cd0>
app = <[AttributeError("'NoneType' object has no attribute 'name'") raised in repr()] SphinxTestApp object at 0x7f6bee651610>, extname = 'rinoh.frontend.sphinx'

    def load_extension(self, app: "Sphinx", extname: str) -> None:
        """Load a Sphinx extension."""
        if extname in app.extensions:  # already loaded
            return
        if extname in EXTENSION_BLACKLIST:
            logger.warning(__('the extension %r was already merged with Sphinx since '
                              'version %s; this extension is ignored.'),
                           extname, EXTENSION_BLACKLIST[extname])
            return

        # update loading context
        prefix = __('while setting up extension %s:') % extname
        with prefixed_warnings(prefix):
            try:
                mod = import_module(extname)
            except ImportError as err:
                logger.verbose(__('Original exception:\n') + traceback.format_exc())
>               raise ExtensionError(__('Could not import extension %s') % extname,
                                     err) from err
E               sphinx.errors.ExtensionError: Could not import extension rinoh.frontend.sphinx (exception: No module named 'rinoh')

/usr/lib/python3.8/site-packages/sphinx/registry.py:428: ExtensionError
============================================================================= warnings summary =============================================================================
tests/test_build.py:28
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.3.1/tests/test_build.py:28: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.noautobuild

tests/test_build.py:83
  /home/tkloczko/rpmbuild/BUILD/sphinx-tabs-3.3.1/tests/test_build.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.noautobuild - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.noautobuild

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_build.py:57: Test uses Sphinx 3 code blocks
ERROR tests/test_build.py::test_rinohtype_pdf - sphinx.errors.ExtensionError: Could not import extension rinoh.frontend.sphinx (exception: No module named 'rinoh')
============================================================ 13 passed, 1 skipped, 2 warnings, 1 error in 2.83s ============================================================

Testest just 3.4.0 and pytest look OK now. Thank you 👍

executablebooks / sphinx-tabs

3.1.0: pytest is failing #126