Running tests and PYTHONPATH

[hsauro]

Is there a simpler way to run the tests that avoids a user having hard code a path in PYTHONPATH?

[joseph-hellerstein]

One possibility is to create a script that runs the tests that gets installed after running setup.py (and so would be installed after running pip install SBstoat). Your thoughts?

[hsauro]

Yesterday I set up something for simplesbml where one can now run the tests by typing

simplesbml.tests.run()

Without having to touch any path variables. That may or may not be what you want with sbstoat depending on ones philosophy on how tests should be handled.

On Tue, Sep 15, 2020 at 9:12 PM Joseph Hellerstein notifications@github.com wrote:

One possibility is to create a script that runs the tests that gets installed after running setup.py (and so would be installed after running pip install SBstoat). Your thoughts?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693158201, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDQFRNMYEJ34MDIZH43SGA3I5ANCNFSM4RL7YJ7A .

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

[CiaranWelsh]

If by "run the tests", you mean running tests for SBstoat, rather than model unit tests configured using sbstoat, what I usually do is ensure I've added the path to my package at the top of each unit test source file. That way it only needs to be done once and by the developer who writes the test. There are two ways I know of to do this which I think are equivalent: 1) sys.path which is a python list so you can append or 2) site.addsitedir() which makes the directory behave as if it were already site-package. Notably, some IDE's (pycharm) will automatically do this for you, but others do not so its more robust to manually add to path.

This is the approach I've taken here lines 6 - 12, though its not ideal because it breaks integration with the IDE.

[hsauro]

I mean running the tests that Joe put together, those are the only tests there are. Currently, you have to manually add the path to the tests which isn't optimal, plus I don't want to keep adding entries to my pythopath if there is a way to avoid it. SBstoat should be self-contained without having to any post configuration. In simplesbml I know where the tests are relative to simplsbml. There were messages on Stockoverflow that showed me how to access packages relative to other packages. Looking at your code I think I'm doing something similar. It now means there is no post configuration and all testing can be handled from simplesbml itself. I like the fact that I can use simplesbml.tests.run() any time I want to check that nothing is broken.

On Wed, Sep 16, 2020 at 12:21 AM Ciaran Welsh notifications@github.com wrote:

If by "run the tests", you mean running tests for SBstoat, rather than model unit tests configured using sbstoat, what I usually do is ensure I've added the path to my package at the top of each unit test source file. That way it only needs to be done once and by the developer who writes the test. There are two ways I know of to do this which I think are equivalent: 1) sys.path which is a python list so you can append or 2) site.addsitedir() which makes the directory behave as if it were already site-package. Notably, some IDE's (pycharm) will automatically do this for you, but others do not so its more robust to manually add to path.

This is the approach I've taken here https://github.com/sys-bio/libOmexMeta/blob/master/tests/python/api_tests.py lines 6 - 12.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693223952, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDXKAJXFP7XXRPI7RLLSGBRPLANCNFSM4RL7YJ7A .

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

[CiaranWelsh]

Ah I see - so the "normal" way in python is to let the test frameworks do the test discovery for you. Any method in a class that inherits from unittest.TestCase that starts with the word "test" shoud be discovered automatically. There are three main test frameworks that come to mind (could be more): pytest, nose and tox.

Running "pytest" from a project should theoretically discover all tests, run them and report to console. I tried with SBstoat and it didn't work. Pytest has a "pytest.ini" configuration file you can place in your project to specify project settings, so my guess is we could get it working.

Running "nosetests" should theoretically do the same thing as "pytest". Actually, I tried this command in SBstoad and the tests are discovered, but all fail with "ModuleNotFoundError: No module named 'docstring_expander'" - probably the path issue. Nose also has a configuration file where I'm guessing you could add paths.

I've also used tox in the past, but not as much because each project needs its own configuration file, rather than just being automatic. However, after the initial configuration it seems to work well.

After all this, I actually do not normally use any of these methods, not directly at least. Pycharm has full integration with each of these testing frameworks and makes it easy to run one test case, a test suite or run them all tests in a directory (they're not paying me for promotion, I promise =]).

[hsauro]

docstring_expander is a dependency and it was not included in the last pypi update. That should be fixed in the next upload to pypi which I might do today.

What would someone actually do to run the tests if it all works properly? Would they have to drop out of the python console and run the tests from the terminal command line? What would the command actually be to runt he tests?

On Wed, Sep 16, 2020 at 10:08 AM Ciaran Welsh notifications@github.com wrote:

Ah I see - so the "normal" way in python is to let the test frameworks do the test discovery for you. Any method in a class that inherits from unittest.TestCase that starts with the word "test" shoud be discovered automatically. There are three main frameworks, pytest, nose and tox.

Running "pytest" from a project should theoretically discover all tests, run them and report to console. I tried with SBstoat and it didn't work. Pytest has a "pytest.ini" configuration file you can place in your project to specify project settings, so my guess is we could get it working.

Running "nosetests" should theoretically do the same thing as "pytest". Actually, I tried this command in SBstoad and the tests are discovered, but all fail with "ModuleNotFoundError: No module named 'docstring_expander'". Nose also has a configuration file, but I can't remember what its called (.noserc or something ).

I've also used tox in the past, but not as much because each project needs its own configuration file, rather than just being automatic. However, after the initial configuration https://tox.readthedocs.io/en/latest/config.html it seems to work well.

After all this, I actually do not normally use any of these methods, not directly at least. Pycharm has full integration https://www.jetbrains.com/help/pycharm/performing-tests.html with each of these testing frameworks and makes it easy to run one test case, a test suite or run them all tests in a directory (they're not paying me for promotion, I promise =]).

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693539056, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDW4IZICZGYKKADQWKLSGDV7RANCNFSM4RL7YJ7A .

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

[hsauro]

PS Joe used nose as the test framework.

On Wed, Sep 16, 2020 at 10:29 AM Herbert M Sauro hsauro@uw.edu wrote:

docstring_expander is a dependency and it was not included in the last pypi update. That should be fixed in the next upload to pypi which I might do today.

What would someone actually do to run the tests if it all works properly? Would they have to drop out of the python console and run the tests from the terminal command line? What would the command actually be to runt he tests?

On Wed, Sep 16, 2020 at 10:08 AM Ciaran Welsh notifications@github.com wrote:

Ah I see - so the "normal" way in python is to let the test frameworks do the test discovery for you. Any method in a class that inherits from unittest.TestCase that starts with the word "test" shoud be discovered automatically. There are three main frameworks, pytest, nose and tox.

Running "pytest" from a project should theoretically discover all tests, run them and report to console. I tried with SBstoat and it didn't work. Pytest has a "pytest.ini" configuration file you can place in your project to specify project settings, so my guess is we could get it working.

Running "nosetests" should theoretically do the same thing as "pytest". Actually, I tried this command in SBstoad and the tests are discovered, but all fail with "ModuleNotFoundError: No module named 'docstring_expander'". Nose also has a configuration file, but I can't remember what its called (.noserc or something ).

I've also used tox in the past, but not as much because each project needs its own configuration file, rather than just being automatic. However, after the initial configuration https://tox.readthedocs.io/en/latest/config.html it seems to work well.

After all this, I actually do not normally use any of these methods, not directly at least. Pycharm has full integration https://www.jetbrains.com/help/pycharm/performing-tests.html with each of these testing frameworks and makes it easy to run one test case, a test suite or run them all tests in a directory (they're not paying me for promotion, I promise =]).

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693539056, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDW4IZICZGYKKADQWKLSGDV7RANCNFSM4RL7YJ7A .

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

[CiaranWelsh]

nose.run() seems to work

(base) PS C:\Users\Ciaran\tmp\SBstoat> pwd

Path
----
C:\Users\Ciaran\tmp\SBstoat

(base) PS C:\Users\Ciaran\tmp\SBstoat> ls

    Directory: C:\Users\Ciaran\tmp\SBstoat

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        16/09/2020     18:27                .idea
d-----        16/09/2020     17:47                .pytest_cache
d-----        16/09/2020     17:47                notebooks
d-----        16/09/2020     18:19                SBstoat
d-----        16/09/2020     17:47                tests
-a----        16/09/2020     17:47           1936 .gitignore
-a----        16/09/2020     17:47            439 .travis.yml
-a----        16/09/2020     17:47            274 jn
-a----        16/09/2020     17:47           1090 LICENSE
-a----        16/09/2020     17:47             49 MANIFEST.in
-a----        16/09/2020     17:47            353 profile.sh
-a----        16/09/2020     17:47           4568 README.md
-a----        16/09/2020     17:47             88 requirements.txt
-a----        16/09/2020     17:47          95913 SBstoat_logo.png
-a----        16/09/2020     17:47             41 setup.cfg
-a----        16/09/2020     17:47           1832 setup.py

(base) PS C:\Users\Ciaran\tmp\SBstoat> ipython
Python 3.7.4 (default, Aug  9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import nose

In [2]: nose.run()

(outputs a slew of failed tests =])

[hsauro]

I'll give it a go today, if it works for me as well I'll update the readme.md with the instructions.

Herbert

On Wed, Sep 16, 2020 at 10:37 AM Ciaran Welsh notifications@github.com wrote:

nose.run() seems to work

(base) PS C:\Users\Ciaran\tmp\SBstoat> pwd

Path

C:\Users\Ciaran\tmp\SBstoat

(base) PS C:\Users\Ciaran\tmp\SBstoat> ls

Directory: C:\Users\Ciaran\tmp\SBstoat

Mode LastWriteTime Length Name

---

d----- 16/09/2020 18:27 .idea d----- 16/09/2020 17:47 .pytest_cache d----- 16/09/2020 17:47 notebooks d----- 16/09/2020 18:19 SBstoat d----- 16/09/2020 17:47 tests -a---- 16/09/2020 17:47 1936 .gitignore -a---- 16/09/2020 17:47 439 .travis.yml -a---- 16/09/2020 17:47 274 jn -a---- 16/09/2020 17:47 1090 LICENSE -a---- 16/09/2020 17:47 49 MANIFEST.in -a---- 16/09/2020 17:47 353 profile.sh -a---- 16/09/2020 17:47 4568 README.md -a---- 16/09/2020 17:47 88 requirements.txt -a---- 16/09/2020 17:47 95913 SBstoat_logo.png -a---- 16/09/2020 17:47 41 setup.cfg -a---- 16/09/2020 17:47 1832 setup.py

(base) PS C:\Users\Ciaran\tmp\SBstoat> ipython Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] Type 'copyright', 'credits' or 'license' for more information IPython 7.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import nose

In [2]: nose.run()

(outputs a slew of failed tests =])

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693554920, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDQATVZMX3BY7LPKML3SGDZTLANCNFSM4RL7YJ7A .

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org hsauro@uw.edu Books: http://books.analogmachine.org/

[joseph-hellerstein]

Another approach is to have setup create a script that runs tests. We could call the script testSBstoat. So, after the pip install, the user enters at the command line: testSBstoat . Command line scripts are easy to run from spyder, jupyter, pycharm, ...

On Wed, Sep 16, 2020, 12:22 AM Ciaran Welsh notifications@github.com wrote:

If by "run the tests", you mean running tests for SBstoat, rather than model unit tests configured using sbstoat, what I usually do is ensure I've added the path to my package at the top of each unit test source file. That way it only needs to be done once and by the developer who writes the test. There are two ways I know of to do this which I think are equivalent: 1) sys.path which is a python list so you can append or 2) site.addsitedir() which makes the directory behave as if it were already site-package. Notably, some IDE's (pycharm) will automatically do this for you, but others do not so its more robust to manually add to path.

This is the approach I've taken here https://github.com/sys-bio/libOmexMeta/blob/master/tests/python/api_tests.py lines 6 - 12.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/sys-bio/SBstoat/issues/3#issuecomment-693223952, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVJIRJCLFGBWA2VUECJGF3SGBRPLANCNFSM4RL7YJ7A .