search

Pylons / webtest

Wraps any WSGI application and makes it easy to send test requests to that application, without starting up an HTTP server.
https://docs.pylonsproject.org/projects/webtest/en/latest/
Other
335 stars 109 forks source link

Switch from Travis to GitHub Actions #231

Closed stevepiercy closed 3 years ago

stevepiercy commented 3 years ago

See also #230.

stevepiercy commented 3 years ago

YES! Coverage is now working properly with GitHub actions. See https://github.com/Pylons/webtest/runs/1389849103?check_suite_focus=true#step:5:283

stevepiercy commented 3 years ago

I tried to add in macos and windows, but I give up. I'm not deeply familiar with the requirements on these OSes to get the builds to pass. It would have been a nice addition.

Anyway, this PR is now ready for review.

digitalresistor commented 3 years ago

The windows errors are due to git automatically changing the line ending on files to match the system it is being checked out on. So on Windows it will silently turn \n into \r\n.

This issue was also happening in Pyramid when testing on Windows, you can use a .gitattributes file to fix this:

https://github.com/Pylons/pyramid/blob/master/.gitattributes

For example.

stevepiercy commented 3 years ago

The windows errors are due to git automatically changing the line ending on files to match the system it is being checked out on. So on Windows it will silently turn \n into \r\n.

This issue was also happening in Pyramid when testing on Windows, you can use a .gitattributes file to fix this:

https://github.com/Pylons/pyramid/blob/master/.gitattributes

I never would have figured that out. Thank you! Fixed in 8e3a168.

That leaves coverage as the sole failing build, which is fine for this PR.

gawel commented 3 years ago

Hi, nice work. Can you explain why it's ok for the coverage to fail ? (logs are no longer available)

stevepiercy commented 3 years ago

Coverage has been broken for at least 7 years. This PR exposes the coverage failures that previously were failing but were reported as passing because there was no --fail-under=100 argument in the automated testing.

That said, the missing coverage should be added in a separate PR. This PR only exposes that coverage was missing.

digitalresistor commented 3 years ago

The logs for the test run are still available (latest on this PR):

2020-11-13T22:32:45.5479953Z Name                  Stmts   Miss  Cover   Missing
2020-11-13T22:32:45.5483892Z ---------------------------------------------------
2020-11-13T22:32:45.5484203Z webtest/__init__.py      15      0   100%
2020-11-13T22:32:45.5484535Z webtest/app.py          337      2    99%   271, 450
2020-11-13T22:32:45.5484864Z webtest/compat.py        17      0   100%
2020-11-13T22:32:45.5485228Z webtest/debugapp.py      60      0   100%
2020-11-13T22:32:45.5485575Z webtest/ext.py            3      0   100%
2020-11-13T22:32:45.5486190Z webtest/forms.py        352     12    97%   90, 117, 252, 273, 278, 287, 465, 472, 572-576
2020-11-13T22:32:45.5486549Z webtest/http.py          85      0   100%
2020-11-13T22:32:45.5486922Z webtest/lint.py         225     27    88%   315, 325, 329, 346, 352, 357, 371, 376, 382, 388, 394, 403, 412, 420, 425, 431, 438, 443, 463, 469, 484, 491, 496, 499, 509, 549, 562
2020-11-13T22:32:45.5487509Z webtest/response.py     232      5    98%   456, 485-491
2020-11-13T22:32:45.5487878Z webtest/sel.py            6      0   100%
2020-11-13T22:32:45.5488417Z webtest/utils.py        101      7    93%   41-42, 98, 118, 130, 133, 166
2020-11-13T22:32:45.5488882Z ---------------------------------------------------
2020-11-13T22:32:45.5489123Z TOTAL                  1433     53    96%
2020-11-13T22:32:45.5489691Z Coverage failure: total of 96 is less than fail-under=100
stevepiercy commented 3 years ago

The logs for the test run are still available (latest on this PR):

All log history prior to the switch to travis-ci.com are available under the travis-ci.org domain:

https://travis-ci.org/github/Pylons/webtest/builds

gawel commented 3 years ago

I see, thanks. Not sure we'll be back to 100% one day...

digitalresistor commented 3 years ago

Before this is merged, if getting back to 100% is not a requirement, the fail flag should be removed.

stevepiercy commented 3 years ago

Instead of its removal, I could change it to --fail-under=96 to prevent further erosion of coverage. Then as coverage improves through PRs, we can bump it up.

I could also try to steer contributors to WebTest as a good first PR. The upcoming PloneConf sprints could be one opportunity.

@gawel it's your call.

gawel commented 3 years ago

@stevepiercy +1 for the 96% option

stevepiercy commented 3 years ago

@gawel done! Coverage is now "passing" at 96%.