search

executablebooks / sphinx-tabs

Tabbed views for Sphinx
https://sphinx-tabs.readthedocs.io
MIT License
264 stars 68 forks source link

Pygments 2.10 tets failures #134

Closed hroncok closed 3 years ago

hroncok commented 3 years ago

Describe the problem

Hello, in Fedora, we want to update Pygments to 2.10, but we see some test failures in sphinx-tabs.

Link to your repository or website

No response

Steps to reproduce

First, I've edited tox.ini to add py39.

$ tox -e py39-sphinx4
GLOB sdist-make: .../sphinx-tabs/setup.py
py39-sphinx4 inst-nodeps: .../sphinx-tabs/.tox/.tmp/package/1/sphinx-tabs-3.2.0.zip
py39-sphinx4 installed: alabaster==0.7.12,appdirs==1.4.4,atomicwrites==1.4.0,attrs==21.2.0,Babel==2.9.1,beautifulsoup4==4.9.3,bs4==0.0.1,certifi==2021.5.30,charset-normalizer==2.0.4,commonmark==0.9.1,coverage==5.5,docutils==0.16,idna==3.2,imagesize==1.2.0,Jinja2==3.0.1,MarkupSafe==2.0.1,more-itertools==8.8.0,packaging==21.0,pluggy==0.13.1,py==1.10.0,Pygments==2.10.0,pyparsing==2.4.7,pytest==3.10.1,pytest-cov==2.9.0,pytest-datadir==1.3.1,pytest-regressions==2.2.0,pytz==2021.1,PyYAML==5.4.1,recommonmark==0.7.1,requests==2.26.0,rinoh-typeface-dejavuserif==0.1.3,rinoh-typeface-texgyrecursor==0.1.1,rinoh-typeface-texgyreheros==0.1.1,rinoh-typeface-texgyrepagella==0.1.1,rinohtype==0.5.3,six==1.16.0,snowballstemmer==2.1.0,soupsieve==2.2.1,Sphinx==4.1.2,sphinx-tabs @ file://.../sphinx-tabs/.tox/.tmp/package/1/sphinx-tabs-3.2.0.zip,sphinx-testing==1.0.1,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,urllib3==1.26.6
py39-sphinx4 run-test-pre: PYTHONHASHSEED='3917563027'
py39-sphinx4 run-test: commands[0] | pytest
============================= test session starts ==============================
platform linux -- Python 3.9.6, pytest-3.10.1, py-1.10.0, pluggy-0.13.1
rootdir: .../sphinx-tabs, inifile:
plugins: datadir-1.3.1, cov-2.9.0, regressions-2.2.0
collected 15 items

tests/test_build.py ..EEE.FFsE.....                                      [100%]

==================================== ERRORS ====================================
_______________ ERROR at setup of test_conditional_assets[index] _______________

app = <SphinxTestApp buildername='html'>
status = <_io.StringIO object at 0x7f8ebb916430>
warning = <_io.StringIO object at 0x7f8ebb916f70>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebb8b34c0>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebb8b31f0>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebb8b3040>
request = <SubRequest 'auto_build_and_check' for <Function 'test_conditional_assets[index]'>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'index.html', encoding = 'utf-8', replace = None

    def read(
        app, buildername="html", filename="index.html", encoding="utf-8", replace=None
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")
            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()
            for find, rep in (replace or {}).items():
                doc = text.replace(find, rep)
        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.html
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.obtained.html
E       HTML DIFF: /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_index_0/test_build/test_conditional_assets_index_.obtained.diff.html
E       --- 
E       +++ 
E       @@ -110,7 +110,7 @@
E             <div aria-labelledby="tab-2-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-2-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -118,7 +118,7 @@
E             <div aria-labelledby="tab-2-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-2-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span><span class="o">++</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="o">++</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -180,9 +180,9 @@
E             <div aria-labelledby="tab-3-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-3-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -190,9 +190,9 @@
E             <div aria-labelledby="tab-3-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-3-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>

tests/conftest.py:91: AssertionError
_____________ ERROR at setup of test_conditional_assets[no_tabs1] ______________

app = <SphinxTestApp buildername='html'>
status = <_io.StringIO object at 0x7f8ebb8fc160>
warning = <_io.StringIO object at 0x7f8ebb8fc4c0>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebb9d5820>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebb9d5700>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebb9d55e0>
request = <SubRequest 'auto_build_and_check' for <Function 'test_conditional_assets[no_tabs1]'>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'index.html', encoding = 'utf-8', replace = None

    def read(
        app, buildername="html", filename="index.html", encoding="utf-8", replace=None
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")
            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()
            for find, rep in (replace or {}).items():
                doc = text.replace(find, rep)
        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs1_.html
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs1_.obtained.html
E       HTML DIFF: /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab0/test_build/test_conditional_assets_no_tabs1_.obtained.diff.html
E       --- 
E       +++ 
E       @@ -110,7 +110,7 @@
E             <div aria-labelledby="tab-2-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-2-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -118,7 +118,7 @@
E             <div aria-labelledby="tab-2-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-2-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span><span class="o">++</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="o">++</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -180,9 +180,9 @@
E             <div aria-labelledby="tab-3-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-3-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -190,9 +190,9 @@
E             <div aria-labelledby="tab-3-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-3-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>

tests/conftest.py:91: AssertionError
_____________ ERROR at setup of test_conditional_assets[no_tabs2] ______________

app = <SphinxTestApp buildername='html'>
status = <_io.StringIO object at 0x7f8ebc01c160>
warning = <_io.StringIO object at 0x7f8ebc090550>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebba39ca0>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebba39f70>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebba398b0>
request = <SubRequest 'auto_build_and_check' for <Function 'test_conditional_assets[no_tabs2]'>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'index.html', encoding = 'utf-8', replace = None

    def read(
        app, buildername="html", filename="index.html", encoding="utf-8", replace=None
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")
            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()
            for find, rep in (replace or {}).items():
                doc = text.replace(find, rep)
        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs2_.html
E       /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs2_.obtained.html
E       HTML DIFF: /tmp/pytest-of-churchyard/pytest-12/test_conditional_assets_no_tab1/test_build/test_conditional_assets_no_tabs2_.obtained.diff.html
E       --- 
E       +++ 
E       @@ -110,7 +110,7 @@
E             <div aria-labelledby="tab-2-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-2-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -118,7 +118,7 @@
E             <div aria-labelledby="tab-2-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-2-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="n">C</span><span class="o">++</span> <span class="n">Main</span> <span class="n">Function</span>
E       +        <pre><span></span><span class="n">C</span><span class="o">++</span><span class="w"> </span><span class="n">Main</span><span class="w"> </span><span class="n">Function</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -180,9 +180,9 @@
E             <div aria-labelledby="tab-3-Qw==" class="sphinx-tabs-panel code-tab group-tab" id="panel-3-Qw==" name="Qw==" role="tabpanel" tabindex="0">
E              <div class="highlight-c notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>
E       @@ -190,9 +190,9 @@
E             <div aria-labelledby="tab-3-Qysr" class="sphinx-tabs-panel code-tab group-tab" hidden="true" id="panel-3-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E              <div class="highlight-c++ notranslate">
E               <div class="highlight">
E       -        <pre><span></span><span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
E       -  <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
E       -<span class="p">}</span>
E       +        <pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="kt">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
E       +<span class="w">  </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
E       +<span class="p">}</span><span class="w"></span>
E        </pre>
E               </div>
E              </div>

tests/conftest.py:91: AssertionError
______________ ERROR at setup of test_other_with_assets_new_style ______________

app = <SphinxTestApp buildername='html'>
status = <_io.StringIO object at 0x7f8ebb548040>
warning = <_io.StringIO object at 0x7f8ebb5480d0>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebb548dc0>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebb548ee0>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebb575040>
request = <SubRequest 'auto_build_and_check' for <Function 'test_other_with_assets_new_style'>>

    @pytest.fixture(autouse=True)
    def auto_build_and_check(
        app,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        request,
    ):
        """
        Build and check build success and output regressions.
        Currently all tests start with this.
        Disable using a `noautobuild` mark.
        """
        if "noautobuild" in request.keywords:
            return
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
>       regress_sphinx_app_output(app)

tests/conftest.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'index.html', encoding = 'utf-8', replace = None

    def read(
        app, buildername="html", filename="index.html", encoding="utf-8", replace=None
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        if buildername == "html":
            soup = BeautifulSoup(content, "html.parser")
            doc_div = soup.findAll("div", {"class": "documentwrapper"})[0]
            doc = doc_div.prettify()
            for find, rep in (replace or {}).items():
                doc = text.replace(find, rep)
        else:
            doc = content
>       file_regression.check(
            doc, extension="." + filename.split(".")[-1], encoding="utf8"
        )
E       AssertionError: FILES DIFFER:
E       /tmp/pytest-of-churchyard/pytest-12/test_other_with_assets_new_sty0/test_build/test_other_with_assets_new_style.html
E       /tmp/pytest-of-churchyard/pytest-12/test_other_with_assets_new_sty0/test_build/test_other_with_assets_new_style.obtained.html
E       HTML DIFF: /tmp/pytest-of-churchyard/pytest-12/test_other_with_assets_new_sty0/test_build/test_other_with_assets_new_style.obtained.diff.html
E       --- 
E       +++ 
E       @@ -13,7 +13,7 @@
E            <div aria-labelledby="tab-0-Qysr" class="sphinx-tabs-panel code-tab group-tab" id="panel-0-Qysr" name="Qysr" role="tabpanel" tabindex="0">
E             <div class="highlight-c++ notranslate">
E              <div class="highlight">
E       -       <pre><span></span><span class="linenos">1</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">"hello world"</span> <span class="o">&lt;&lt;</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
E       +       <pre><span></span><span class="linenos">1</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="w"> </span><span class="o">&lt;&lt;</span><span class="w"> </span><span class="s">"hello world"</span><span class="w"> </span><span class="o">&lt;&lt;</span><span class="w"> </span><span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span><span class="w"></span>
E        </pre>
E              </div>
E             </div>

tests/conftest.py:91: AssertionError
=================================== FAILURES ===================================
_____________ test_conditional_assets_html_assets_policy[no_tabs1] _____________

app = <SphinxTestApp buildername='html'>, docname = 'no_tabs1'
status = <_io.StringIO object at 0x7f8ebb75adc0>
warning = <_io.StringIO object at 0x7f8ebbcaf160>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebb51f1f0>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebb75ae50>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebb75aca0>
check_asset_links = <function check_asset_links.<locals>.check at 0x7f8ebb75aaf0>

    @pytest.mark.noautobuild
    @pytest.mark.parametrize("docname", ["index", "no_tabs1", "no_tabs2"])
    @pytest.mark.sphinx(testroot="conditionalassets")
    @pytest.mark.skipif(
        sphinx.version_info[:2] < (4, 1), reason="Test uses Sphinx 4.1 config"
    )
    def test_conditional_assets_html_assets_policy(
        app,
        docname,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        check_asset_links,
    ):
        app.set_html_assets_policy("always")

        # Following lines are copied from ``auto_build_and_check`` since we need to
        # set a config in the build object before auto build. Because of this, we
        # need to use ``noautobuild``.
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
        regress_sphinx_app_output(app)

>       check_asset_links(app, filename=docname + ".html")

tests/test_build.py:54: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'no_tabs1.html', encoding = 'utf-8', cssPresent = True
jsPresent = True

    def check(
        app,
        buildername="html",
        filename="index.html",
        encoding="utf-8",
        cssPresent=True,
        jsPresent=True,
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        css_assets = [f for f in FILES if f.endswith(".css")]
        js_assets = [f for f in FILES if f.endswith(".js")]

        soup = BeautifulSoup(content, "html.parser")
        stylesheets = soup.find_all("link", {"rel": "stylesheet"}, href=True)
        css_refs = [s["href"] for s in stylesheets]

        scripts = soup.find_all("script", src=True)
        js_refs = [s["src"] for s in scripts]

        all_refs = css_refs + js_refs

        if cssPresent:
            css_present = all(any(a in ref for ref in all_refs) for a in css_assets)
            assert css_present
        else:
            assert not "sphinx_tabs" in css_refs
        if jsPresent:
            js_present = all(any(a in ref for ref in js_refs) for a in js_assets)
>           assert js_present
E           assert False

tests/conftest.py:161: AssertionError
--------------------------- Captured stdout teardown ---------------------------
# testroot: root
# builder: html
# srcdir: /tmp/pytest-of-churchyard/pytest-12/conditionalassets
# outdir: /tmp/pytest-of-churchyard/pytest-12/conditionalassets/_build/html
# status: 
Running Sphinx v4.1.2
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.
build succeeded.

The HTML pages are in ../../../../../tmp/pytest-of-churchyard/pytest-12/conditionalassets/_build/html.

# warning: 

_____________ test_conditional_assets_html_assets_policy[no_tabs2] _____________

app = <SphinxTestApp buildername='html'>, docname = 'no_tabs2'
status = <_io.StringIO object at 0x7f8ebb9740d0>
warning = <_io.StringIO object at 0x7f8ebb9741f0>
check_build_success = <function check_build_success.<locals>.check at 0x7f8ebb51f940>
get_sphinx_app_doctree = <function get_sphinx_app_doctree.<locals>.read at 0x7f8ebb51fa60>
regress_sphinx_app_output = <function regress_sphinx_app_output.<locals>.read at 0x7f8ebb51fb80>
check_asset_links = <function check_asset_links.<locals>.check at 0x7f8ebb51fc10>

    @pytest.mark.noautobuild
    @pytest.mark.parametrize("docname", ["index", "no_tabs1", "no_tabs2"])
    @pytest.mark.sphinx(testroot="conditionalassets")
    @pytest.mark.skipif(
        sphinx.version_info[:2] < (4, 1), reason="Test uses Sphinx 4.1 config"
    )
    def test_conditional_assets_html_assets_policy(
        app,
        docname,
        status,
        warning,
        check_build_success,
        get_sphinx_app_doctree,
        regress_sphinx_app_output,
        check_asset_links,
    ):
        app.set_html_assets_policy("always")

        # Following lines are copied from ``auto_build_and_check`` since we need to
        # set a config in the build object before auto build. Because of this, we
        # need to use ``noautobuild``.
        app.build()
        check_build_success(status, warning)
        get_sphinx_app_doctree(app, regress=True)
        regress_sphinx_app_output(app)

>       check_asset_links(app, filename=docname + ".html")

tests/test_build.py:54: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

app = <SphinxTestApp buildername='html'>, buildername = 'html'
filename = 'no_tabs2.html', encoding = 'utf-8', cssPresent = True
jsPresent = True

    def check(
        app,
        buildername="html",
        filename="index.html",
        encoding="utf-8",
        cssPresent=True,
        jsPresent=True,
    ):
        content = get_sphinx_app_output(app, buildername, filename, encoding)

        css_assets = [f for f in FILES if f.endswith(".css")]
        js_assets = [f for f in FILES if f.endswith(".js")]

        soup = BeautifulSoup(content, "html.parser")
        stylesheets = soup.find_all("link", {"rel": "stylesheet"}, href=True)
        css_refs = [s["href"] for s in stylesheets]

        scripts = soup.find_all("script", src=True)
        js_refs = [s["src"] for s in scripts]

        all_refs = css_refs + js_refs

        if cssPresent:
            css_present = all(any(a in ref for ref in all_refs) for a in css_assets)
            assert css_present
        else:
            assert not "sphinx_tabs" in css_refs
        if jsPresent:
            js_present = all(any(a in ref for ref in js_refs) for a in js_assets)
>           assert js_present
E           assert False

tests/conftest.py:161: AssertionError
--------------------------- Captured stdout teardown ---------------------------
# testroot: root
# builder: html
# srcdir: /tmp/pytest-of-churchyard/pytest-12/conditionalassets
# outdir: /tmp/pytest-of-churchyard/pytest-12/conditionalassets/_build/html
# status: 
Running Sphinx v4.1.2
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.
build succeeded.

The HTML pages are in ../../../../../tmp/pytest-of-churchyard/pytest-12/conditionalassets/_build/html.

# warning: 

============ 2 failed, 8 passed, 1 skipped, 4 error in 4.75 seconds ============
ERROR: InvocationError for command .../sphinx-tabs/.tox/py39-sphinx4/bin/pytest (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py39-sphinx4: commands failed

The version of Python you're using

3.9.6, 3.10.0rc1

Your operating system

Fedora Linux 33-36

Versions of your packages

Package                       Version
----------------------------- ---------
alabaster                     0.7.12
appdirs                       1.4.4
atomicwrites                  1.4.0
attrs                         21.2.0
Babel                         2.9.1
beautifulsoup4                4.9.3
bs4                           0.0.1
certifi                       2021.5.30
charset-normalizer            2.0.4
commonmark                    0.9.1
coverage                      5.5
docutils                      0.16
idna                          3.2
imagesize                     1.2.0
Jinja2                        3.0.1
MarkupSafe                    2.0.1
more-itertools                8.8.0
packaging                     21.0
pip                           21.1.1
pluggy                        0.13.1
py                            1.10.0
Pygments                      2.10.0
pyparsing                     2.4.7
pytest                        3.10.1
pytest-cov                    2.9.0
pytest-datadir                1.3.1
pytest-regressions            2.2.0
pytz                          2021.1
PyYAML                        5.4.1
recommonmark                  0.7.1
requests                      2.26.0
rinoh-typeface-dejavuserif    0.1.3
rinoh-typeface-texgyrecursor  0.1.1
rinoh-typeface-texgyreheros   0.1.1
rinoh-typeface-texgyrepagella 0.1.1
rinohtype                     0.5.3
setuptools                    56.1.0
six                           1.16.0
snowballstemmer               2.1.0
soupsieve                     2.2.1
Sphinx                        4.1.2
sphinx-tabs                   3.2.0
sphinx-testing                1.0.1
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
urllib3                       1.26.6
wheel                         0.36.2

Additional context

No response

welcome[bot] commented 3 years ago

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

foster999 commented 3 years ago

Hi @hroncok, thanks for raising this :)

As our pygments version isn't fixed, I find that regression tests for code that wraps around code-block break each time pygments changes. Parameterizing tests on pygments version should sort this out, but I'd be happy to hear any other suggestions!

foster999 commented 3 years ago

The PR above should get the tests passing again, once a related issue has been sorted there. Worth noting that the current failures aren't caused by anything functional, as our code-tabs just wrap around pygments.

foster999 commented 3 years ago

@hroncok tests are working and now check against a few pygments versions (inc 2.10). Does this seem alright from your end?

hroncok commented 3 years ago

Thank you very much! It seems alright.