search

Bachmann1234 / diff_cover

Automatically find diff lines that need test coverage.
Apache License 2.0
713 stars 191 forks source link

Two failing tests on Fedora rawhide #136

Open bowlofeggs opened 4 years ago

bowlofeggs commented 4 years ago

Greetings!

I am seeing two test failures when I run the test suite on Fedora Rawhide. I used git bisect, and this commit seems to be where the two failures were introduced: https://github.com/Bachmann1234/diff_cover/commit/7e9f14d7769978b9f463852dd9f32a70c964780a

I see that the commit message references a change in pycodestyle, but Rawhide has pycodestyle 2.5.0, which seems to be the latest version. Is there perhaps some other dependency that I should look at?


====================================================== FAILURES =======================================================                                                                                                                       
_______________________________ DiffQualityIntegrationTest.test_html_with_external_css ________________________________                                                                                                                       

self = <diff_cover.tests.test_integration.DiffQualityIntegrationTest testMethod=test_html_with_external_css>                                                                                                                                  

    def test_html_with_external_css(self):                                                                                                                                                                                                    
>       temp_dir = self._check_html_report(                                                                                                                                                                                                   
            'git_diff_violations.txt',                                                                                                                                                                                                        
            'pycodestyle_violations_report_external_css.html',                                                                                                                                                                                
            ['diff-quality', '--violations=pycodestyle'],                                                              
            css_file='external_style.css'                  
        )                                                                                                                                                                                                                                     

diff_cover/tests/test_integration.py:440:                  
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
diff_cover/tests/test_integration.py:119: in _check_html_report
    assert_long_str_equal(expected, html, strip=True)    
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

expected = '<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">\n<html>\n    <head>\n     ...pan class="kc">None</span>\n</pre></div>\n</td></tr></table>\n            </div>\n        </div>\n    </bo
dy>\n</html>'                                              
actual = '<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">\n<html>\n    <head>\n     ...pan class="bp">None</span>\n</pre></div>\n</td></tr></table>\n            </div>\n        </div>\n    </body
>\n</html>'                                                                                                            
strip = True                                                                                                           

    def assert_long_str_equal(expected, actual, strip=False):                             
        """                                                                                                                                                                                                                                   
        Assert that two strings are equal and                                                                          
        print the diff if they are not.                                                                                

        If `strip` is True, strip both strings before comparing.                                                                                                                                                                              
        """                                                                                                                                                                                                                                   
        # If we've been given a byte string, we need to convert                                                        
        # it back to unicode.  Otherwise, Python3 won't                                                                                                                                                                                       
        # let us use string methods!                                                                                                                                                                                                          
        if isinstance(expected, six.binary_type):                                                                      
            expected = expected.decode('utf-8')                                                                        
        if isinstance(actual, six.binary_type):                                                                                                                                                                                               
            actual = actual.decode('utf-8')                                                                            

        if strip:                                          
            expected = expected.strip()                    
            actual = actual.strip()                        

        if expected != actual:                             

            # Print a human-readable diff                  
            diff = difflib.Differ().compare(               
                expected.split('\n'), actual.split('\n')                                                               
            )                                                                                                                                                                                                                                 

            # Fail the test                                                                                                                                                                                                                   
>           assert False, '\n\n' + '\n'.join(diff)
E           AssertionError:                                                                                                                                                                                                         [105/1744]
E                                                          
E             <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
E             <html>                                       
E                 <head>                                   
E                     <meta http-equiv='Content-Type' content='text/html; charset=utf-8'>
E                     <title>Diff Quality</title>
E                     <link rel="stylesheet" href="external_style.css">
E                 </head>                                  
E                 <body>                                   
E                     <h1>Diff Quality</h1>
E                     <p>Quality Report: pycodestyle</p>
E                     <p>Diff: origin/master...HEAD, staged and unstaged changes</p>
E                     <table border="1">
E                         <tr>
E                             <th>Source File</th>
E                             <th>Diff Quality (%)</th>
E                             <th>Lines in violation</th>
E                         </tr>
E                         <tr>
E                             <td>violations_test_file.py</td>
E                             <td>66.7%</td>
E                             <td>
E                                 <ul>
E                                     <li>2: E225 missing whitespace around operator</li>
E                                     <li>6: E302 expected 2 blank lines, found 0</li>
E                                     <li>11: E225 missing whitespace around operator</li>
E                                 </ul>
E                             </td>
E                         </tr>
E                     </table>
E                     <ul>                                 
E                         <li><b>Total</b>: 9 lines</li>
E                         <li><b>Violation</b>: 3 lines</li>
E                         <li><b>% Quality</b>: 66%</li>
E                     </ul>                                
E                     <div class="src-snippet">
E                         <div class="src-name">violations_test_file.py</div>
E                         <div class="snippets">
E                         <table class="snippettable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
E              2                                           
E              3                                           
E              4                                           
E              5                                           
E              6                                           
E              7                                           
E              8                                           
E              9                                           
E             10                                           
E             11
E             12</pre></div></td><td class="code"><div class="snippet"><pre><span></span><a name="violations_test_file.py-1"></a><span class="k">def</span> <span class="nf">func_1</span><span class="p">(</span><span class="n">apple</span>
<span class="p">,</span> <span class="n">my_list</span><span class="p">):</span>                                                                                                                                                              
E             <a name="violations_test_file.py-2"></a><span class="hll">    <span class="k">if</span> <span class="n">apple</span><span class="o">&lt;</span><span class="mi">10</span><span class="p">:</span>                               
E             </span><a name="violations_test_file.py-3"></a>        <span class="c1"># Do something </span>           
E             <a name="violations_test_file.py-4"></a>        <span class="n">my_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">apple</span><span class="p">)</span>                 
E             <a name="violations_test_file.py-5"></a>    <span class="k">return</span> <span class="n">my_list</span><span class="p">[</span><span class="mi">1</span><span class="p">:]</span>                                              
E             <a name="violations_test_file.py-6"></a><span class="hll"><span class="k">def</span> <span class="nf">func_2</span><span class="p">(</span><span class="n">spongebob</span><span class="p">,</span> <span class="n">squarepants<
/span><span class="p">):</span>                            
E             </span><a name="violations_test_file.py-7"></a>    <span class="sd">&quot;&quot;&quot;A less messy function&quot;&quot;&quot;</span>                                                                                            
E             <a name="violations_test_file.py-8"></a>    <span class="k">for</span> <span class="n">char</span> <span class="ow">in</span> <span class="n">spongebob</span><span class="p">:</span>                                          
E             <a name="violations_test_file.py-9"></a>        <span class="k">if</span> <span class="n">char</span> <span class="ow">in</span> <span class="n">squarepants</span><span class="p">:</span>                                     
E             <a name="violations_test_file.py-10"></a>            <span class="k">return</span> <span class="n">char</span>                                                                                                                  
E             <a name="violations_test_file.py-11"></a><span class="hll">    <span class="n">unused</span><span class="o">=</span><span class="mi">1</span>                                                                                   
E           - </span><a name="violations_test_file.py-12"></a>    <span class="k">return</span> <span class="kc">None</span>                                                                                                                  
E           ?                                                                                                ^^        
E                                                          
E           + </span><a name="violations_test_file.py-12"></a>    <span class="k">return</span> <span class="bp">None</span>                                                                                                                  
E           ?                                                                                                ^^        
E                                                          
E             </pre></div>                                 
E             </td></tr></table>                           
E                         </div>                         
E                     </div>                               
E                 </body>                                  
E             </html>                                                                                                  

diff_cover/tests/helpers.py:43: AssertionError             
_____________________________________ SnippetLoaderTest.test_load_declared_arabic _____________________________________

self = <diff_cover.tests.test_snippets.SnippetLoaderTest testMethod=test_load_declared_arabic>

    def test_load_declared_arabic(self):                                                                                                                                                                                                      
>       self._compare_snippets_html_output('snippet_8859.py',                                                          
                                           [7],                                                                        
                                           'snippet_arabic_output.html')                                               

diff_cover/tests/test_snippets.py:215:                                                                                                                                                                                                        
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
diff_cover/tests/test_snippets.py:202: in _compare_snippets_html_output                                                                                                                                                                       
    assert_long_str_equal(expected, snippets_html, strip=True)                                                                                                                                                                                
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

expected = '<table class="snippettable"><tr><td class="linenos"><div class="linenodiv"><pre> 3\n 4\n 5\n 6\n 7\n 8\n 9\n10</pre><...name="snippet_8859.py-10"></a><span class="n">Line</span> <span class="mi">10</span>\n</pre></div>\n</td><
/tr></table>'                                                                                                          
actual = '<table class="snippettable"><tr><td class="linenos"><div class="linenodiv"><pre> 3\n 4\n 5\n 6\n 7\n 8\n 9\n10</pre><...name="snippet_8859.py-10"></a><span class="n">Line</span> <span class="mi">10</span>\n</pre></div>\n</td></t
r></table>'                                                
strip = True                                               

    def assert_long_str_equal(expected, actual, strip=False):                                                                                                                                                                                 
        """                                                                                                                                                                                                                                   
        Assert that two strings are equal and                                                                                                                                                                                                 
        print the diff if they are not.                                                                                                                                                                                                       

        If `strip` is True, strip both strings before comparing.                                                                                                                                                                              
        """                                                                                                                                                                                                                                   
        # If we've been given a byte string, we need to convert                                                                                                                                                                               
        # it back to unicode.  Otherwise, Python3 won't                                                                                                                                                                                       
        # let us use string methods!                                                                                   
        if isinstance(expected, six.binary_type):                                                                                                                                                                                             
            expected = expected.decode('utf-8')                                                                                                                                                                                               
        if isinstance(actual, six.binary_type):                                                                                                                                                                                               
            actual = actual.decode('utf-8')                

        if strip:                                                                                                                                                                                                                             
            expected = expected.strip()                                                                                                                                                                                                       
            actual = actual.strip()                                                                                                                                                                                                           

        if expected != actual:                                                                                                                                                                                                                

            # Print a human-readable diff                  
            diff = difflib.Differ().compare(                                                                                                                                                                                                  
                expected.split('\n'), actual.split('\n')                                                               
            )                                              

            # Fail the test                                
>           assert False, '\n\n' + '\n'.join(diff)
E           AssertionError:                                
E                                                          
E             <table class="snippettable"><tr><td class="linenos"><div class="linenodiv"><pre> 3                       
E              4                                           
E              5                                           
E              6                                                                                                       
E              7                                                                                                       
E              8                                                                                                       
E              9                                                                                                       
E             10</pre></div></td><td class="code"><div class="snippet"><pre><span></span><a name="snippet_8859.py-3"></a><span class="n">Line</span> <span class="mi">3</span>                                                                
E             <a name="snippet_8859.py-4"></a><span class="n">Line</span> <span class="mi">4</span>                    
E             <a name="snippet_8859.py-5"></a><span class="n">Line</span> <span class="mi">5</span>                    
E             <a name="snippet_8859.py-6"></a><span class="n">Line</span> <span class="mi">6</span>                    
E           - <a name="snippet_8859.py-7"></a><span class="hll"><span class="n">Line</span> <span class="mi">7</span>  <span class="n">ئئئئئ</span> <span class="n">ئئئئئ</span>                                                              
E           ?                                                                                                                       ^                            ^                                                                            
E                                                                                                                      
E           + <a name="snippet_8859.py-7"></a><span class="hll"><span class="n">Line</span> <span class="mi">7</span>  <span class="err">ئئئئئ</span> <span class="err">ئئئئئ</span>                                                          
E           ?                                                                                                                       ^^^                            ^^^                                                                        
E                                                                                                                      
E             </span><a name="snippet_8859.py-8"></a><span class="n">Line</span> <span class="mi">8</span>             
E             <a name="snippet_8859.py-9"></a><span class="n">Line</span> <span class="mi">9</span>                                                                                                                                           
E             <a name="snippet_8859.py-10"></a><span class="n">Line</span> <span class="mi">10</span>
E             </pre></div>                                 
E             </td></tr></table>

diff_cover/tests/helpers.py:43: AssertionError
======================================== 2 failed, 232 passed in 10.29 seconds ========================================
Bachmann1234 commented 4 years ago

Hmm, I have not been able to reproduce this issue on my mac or in CI

here is a test run and a pip freeze with my mac

py.test .
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /Users/bachmann/code/diff_cover
plugins: cov-2.8.1
collected 234 items

diff_cover/tests/test_args.py .....                                      [  2%]
diff_cover/tests/test_diff_cover_main.py ..                              [  2%]
diff_cover/tests/test_diff_quality_main.py ...........                   [  7%]
diff_cover/tests/test_diff_reporter.py ..............................    [ 20%]
diff_cover/tests/test_git_diff.py .....                                  [ 22%]
diff_cover/tests/test_git_path.py .....                                  [ 24%]
diff_cover/tests/test_integration.py ................................... [ 39%]
........                                                                 [ 43%]
diff_cover/tests/test_java_violations_reporter.py ..............         [ 49%]
diff_cover/tests/test_report_generator.py .....................          [ 58%]
diff_cover/tests/test_snippets.py .................                      [ 65%]
diff_cover/tests/test_violations_reporter.py ........................... [ 76%]
......................................................                   [100%]

============================= 234 passed in 6.69s ==============================

~/code/diff_cover master 7s
venv ❯ pip freeze
astroid==2.3.3
attrs==19.3.0
coverage==5.0.3
entrypoints==0.3
flake8==3.7.9
importlib-metadata==1.4.0
inflect==4.0.0
isort==4.3.21
Jinja2==2.10.3
jinja2-pluralize==0.3.0
lazy-object-proxy==1.4.3
MarkupSafe==1.1.1
mccabe==0.6.1
mock==3.0.5
more-itertools==8.1.0
packaging==20.0
pluggy==0.13.1
py==1.8.1
pycodestyle==2.5.0
pydocstyle==5.0.2
pyflakes==2.1.1
Pygments==2.5.2
pylint==2.4.4
pyparsing==2.4.6
pytest==5.3.4
pytest-cov==2.8.1
six==1.14.0
snowballstemmer==2.0.0
typed-ast==1.4.1
wcwidth==0.1.8
wrapt==1.11.2
zipp==2.0.0
Bachmann1234 commented 4 years ago

Here is a CI run on master. Maybe the logs have a hint https://travis-ci.org/Bachmann1234/diff_cover/builds/640267623