Shows all tests passed instead of fail.

[ev-agelos]

I installed the plugin moved to the standard directory where i run usually py.test(from terminal) in order to see how pytest.vim works. The directory that i usually run the tests is one level up from the tests folder: MyProject/app1/ In a few words the issue here is that i changed a test in order to fail and run :Pytest function, and i am getting green bar with all tests passed, then i type :Pytest session and i see that the test actually failed.

I tried different things because i read the other ticket that depends in what directory you are so:

• From directory: MyProject/app1/
  • Executed: gvim tests/unit/test_something.py Result: green Executed :pwd showed: ...MyProject/app1/
  • Executed: gvim, then inside gvim: :e tests/unit/test_something.py Result: green Executed :pwd showed: ...MyProject/app1/
  • Executed: gvim where i have installed startify plugin which shows the most recent files used so i opened then the same test file using startify Result: red Executed :pwd showed: ...MyProject/app1/tests/unit/
• From directory: MyProject/app1/tests/
  • Executed: gvim unit/test_something.py Result: green Executed :pwd showed: ...MyProject/app1/tests/
  • Executed: gvim, then inside gvim: :e tests/unit/test_something.py Result: green Executed :pwd showed: ...MyProject/app1/tests/
  • Executed: gvim where i have installed startify plugin which shows the most recent files used so i opened then the same test file using startify Result: green Executed :pwd showed: ...MyProject/app1/tests/unit

    Conclusion:

From inside /tests/ folder always shows green(all tests passed) but it shouldnt. From inside my apps folder /app1/ worked only when i opened the file with :e tests/unit/test_something.py

:Pytest session worked as expected in every situaton and showed the failed test.

My project structure is:

• My project/
  • app1/
  • tests/
    • conftest.py
    • unit/
    • test_something.py
    • functional/
    • test_something_else.py
  • app2/....

[alfredodeza]

Let me clarify a few things that you mention:

• pytest.vim does not depend on what directory you are. You can be on '/' and run a test in '$HOME/my_app/tests/test_foo.py`
• You mention that you "executed gvim tests/unit/test_something.py" but that doesn't tell me what you used. Pytest method ? or Pytest file ? something else?

I would need a way to replicate your problem, I would need the contents of your test files, but most importantly, you say you see a green bar but you do see the failure in Pytest session, but you don't show me how that failure looks.

The parsing done to understand if you got a failure or not may be a bit naive, but I can't tell exactly if I don't see the output and you are not giving me a way to replicate.

[ev-agelos]

I read also the other similar ticket and understood that it doesnt depend the directory that u run a test but i wanted to show as much information as possible. I used :Pytest function in every situation(i am sorry i forgot to say that). Well the code is private so i can not show anything from the code but when i run :Pytest session well it shows what Pytest python library show.

Feel free to close that issue as i dont see anyone else complaining about that so maybe im missing something.

[alfredodeza]

@ev-agelos I really want to get this fixed if you are having issues though.

Could you at least give me the output of what Pytest session looks? And the best way to debug this would be for you to try and replicate this on another new project so you can find a way to allow me to debug further.

[ev-agelos]

Again the session shows the same result as i would have run it from terminal using the Python library. Ill try to make the same structure with folders, a new test project in the next days and i hope i will get the same error and give you as much info as possible.

[alfredodeza]

@ev-agelos I require that Pytest session output, even if you consider that to be the same. Because I don't know how it is the same. If you cannot provide me with a paste of the output I cannot debug this.

pytest.vim parses the output, there is something in that output that is making the bar to go green with the generic "All tests passed" and I cannot tell what is causing that until you provide that output.

Please provide me with the output of Pytest session so I can debug this.

[ev-agelos]

Ok here it is. I open the test file from project/app1/

My project structure(the only file that exists is the test_something.py with one function inside that i assert 1 == 2 just to make it fail):

• project/
  • app1/
  • tests/
    • unit/
    • test_something.py
    • functional/
  • app2/
  • tests/
    • unit/
    • functional/

OUTPUT of the :Pytest session

Test session starts (platform: linux2, Python 2.7.6, pytest 2.7.1, pytest-sugar 0.4.0)
rootdir: /home/evagelos/Desktop/project/app1/tests/unit, inifile: 
plugins: sugar, capturelog, raisesregexp, cov

\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015 test_description \u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015\u2015
tests/unit/test_something.py:3: in test_description
    assert 1 == 2
E   assert 1 == 2

\r \x1b[36m\x1b[0mtest_something.py\x1b[0m \x1b[31m\u2a2f\x1b[0m                                             \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m\u2588\x1b[0m\x1b[40m\x1b[32m\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\x1b[0m

Results (0.01s):
       1 failed
         - /home/evagelos/Desktop/project/app1/tests/unit/test_something.py:3: assert 1 == 2

Also, when i created the folders etc, i opened to test it and sometimes it worked as expected! I suddenly understood that when i was saving (:w) and then :Pytest fucntion it worked(it was showing there was an error or something like that with the red bar!). I am trying to do it again and i cant i always now get the green bar, so the only thing i can think of maybe conflicts with another plugin? I really dont know

[alfredodeza]

What are all those codes? ANSI escape codes? That will surely break parsing

[ev-agelos]

Output from terminal:

Test session starts (platform: linux2, Python 2.7.6, pytest 2.7.1, pytest-sugar 0.4.0)
rootdir: /home/evagelos/Desktop/project/app1/tests/unit, inifile: 
plugins: sugar, capturelog, raisesregexp, cov

――――――――――――――――――――――――――――――― test_description ―――――――――――――――――――――――――――――――

    def test_description():
        """Ensure its correct."""
>       assert 1 == 2
E       assert 1 == 2

tests/unit/test_something.py:3: AssertionError

 test_something.py ⨯                                             100% ██████████

Results (0.01s):
       1 failed
         - /home/evagelos/Desktop/project/app1/tests/unit/test_something.py:3: assert 1 == 2

[ev-agelos]

The problem was from sugar plugin. I uninstalled it and worked. And then i read about delgado..(It would be nice to have sugar installed and run tests either from vim or from terminal). I also found another ticket that says the same thing about sugar.

[alfredodeza]

I think that pytest.vim could do better here though. Do you think that if the plugin returned a warning message if it detected the ANSI color codes would be reasonable? Something like:

Could not parse py.test output. If using a plugin that alters the default output, consider disabling it.

[ev-agelos]

For sure anything would be helpful than the green bar + all tests passed :D Also please add to README also that sugar has to be uninstalled etc

Edit: To tell you the truth, if result got altered even by 1 character never show green bar.