Open westurner opened 4 years ago
Not that I'm aware of. Testing is mostly by hand and without automation.
Does the testing automation still need to be added for this project?
I don't think it is needed, but if you want to do it and are willing to keep it running forever, go right ahead. It's a big investment in personal time, and at the rate of change for Pippy (see commits) it hardly seems justified. A step by step guide for human testing might be a better investment. We have a guide now.
It's extremely unlikely that an investment of time into automated testing would be a waste relative to other development activities.
Testing [GUI] apps manually is a huge time suck; and humans skip steps and find bugs not yet present in the test.
Maybe start with coverage on the main()
function? pytest-cov
is one way to check test coverage: what percentage of source code lines and or expressions are executed by at least one test function:
pytest --cov=mypkg --cov-report=term-missing tests/
https://pypi.org/project/pytest-cov/
Is there a specific pytest plugin (e.g. pytest-qt) that the sugarlabs projects prefer to use for testing GUI apps in sugar; because quality assurance?
Manual testing is very slow, expensive, and error prone. For one, automated testing makes it possible to change without breaking other people's code: if it's going to be days or weeks to get a PR merged due to waiting for manual testing to occur, people don't consider contributing.
My idea wasn't to develop the automated testing for just Pippy but rather a general framework from a project that I have worked on for the calculate activity and modify it so it can test different activities for the entire project (since this was one of the first projects that called for a need for automated testing).
Minimally, what does a GUI testing framework have?
I don't think a testing framework would be required. As @quozl mentioned, last commit was made 3 years ago, the frequency of commits is way too little to justify the efforts needed to build a whole testing framework for Pippy. I agree this could be helpful if built to integrate with any sugar activity, even though I don't think many activities have that active commit history, but then this issue shouldn't belong here. What do you think @chimosky
This works with and without pytest (and with python -O
, which strips
assert
keyword statements and docstrings):
import unittest
test = unittest.TestCase()
def test_one():
assert 1 == 1, "one does not equal one!"
test.assertEqual(1, 1, "one doesn't = one")
def main(argv=None):
if '-t' in argv or '--test' in argv:
run_all_tests() # pytest.main
Pytest preprocesses assert
statements in order to upgrade assertion
functions to better error messages.
Typically, we just run $ pytest
in a shell with the correct path; though
there's this trick to make a script self-testing:
if opts.run_tests:
sys.argv = [sys.argv[0]] + args
# return unittest.main()
return subprocess.call(["pytest", "-v"] + args + [__file__])
Further notes on testing: https://github.com/sagemathinc/cocalc-doc/issues/40#issuecomment-1597678379
On Mon, Jul 3, 2023, 10:04 PM Sparsh Goenka @.***> wrote:
I don't think a testing framework would be required. As @quozl https://github.com/quozl mentioned, last commit was made 3 years ago, the frequency of commits is way too little to justify the efforts needed to build a whole testing framework for Pippy. I agree this could be helpful if built to integrate with any sugar activity, even though I don't think many activities have that active commit history, but then this issue won't belong here. What do you think @chimosky https://github.com/chimosky
— Reply to this email directly, view it on GitHub https://github.com/sugarlabs/Pippy/issues/84#issuecomment-1619355402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAMNSY5OOHKFLRYHYYGTSTXON23ZANCNFSM4OXFKZ3Q . You are receiving this because you authored the thread.Message ID: @.***>
A PR to demo testing without pytest or expect: https://github.com/4dsolutions/python_camp/pull/4/files
On Tue, Jul 4, 2023, 8:07 AM Wes Turner @.***> wrote:
This works with and without pytest (and with
python -O
, which stripsassert
keyword statements and docstrings):import unittest test = unittest.TestCase() def test_one(): assert 1 == 1, "one does not equal one!" test.assertEqual(1, 1, "one doesn't = one") def main(argv=None): if '-t' in argv or '--test' in argv: run_all_tests() # pytest.main
Pytest preprocesses
assert
statements in order to upgrade assertion functions to better error messages.Typically, we just run
$ pytest
in a shell with the correct path; though there's this trick to make a script self-testing:if opts.run_tests: sys.argv = [sys.argv[0]] + args # return unittest.main() return subprocess.call(["pytest", "-v"] + args + [__file__])
Further notes on testing: https://github.com/sagemathinc/cocalc-doc/issues/40#issuecomment-1597678379
On Mon, Jul 3, 2023, 10:04 PM Sparsh Goenka @.***> wrote:
I don't think a testing framework would be required. As @quozl https://github.com/quozl mentioned, last commit was made 3 years ago, the frequency of commits is way too little to justify the efforts needed to build a whole testing framework for Pippy. I agree this could be helpful if built to integrate with any sugar activity, even though I don't think many activities have that active commit history, but then this issue won't belong here. What do you think @chimosky https://github.com/chimosky
— Reply to this email directly, view it on GitHub https://github.com/sugarlabs/Pippy/issues/84#issuecomment-1619355402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAMNSY5OOHKFLRYHYYGTSTXON23ZANCNFSM4OXFKZ3Q . You are receiving this because you authored the thread.Message ID: @.***>
I think testing would be of great help but right now except someone wants to take it on the themselves to write tests for this activity first before looking at the broader sugar activities then I'll gladly review it.
@westurner it'll be of great help if you have something you'd like to share.
Are there tests for this software?
Is there a Software QA / Testing plan / recommended toolset for sugar or specifically for pippy?
I see: https://wiki.sugarlabs.org/go/Category:Testing
From https://bugzilla.redhat.com/show_bug.cgi?id=1855939#c1 ::