repobee / repobee-sanitizer

A plugin for sanitizing master repositories before distribution to students
MIT License
2 stars 3 forks source link

No test checking if result message is for create_pull_request #140

Open tohanss opened 3 years ago

tohanss commented 3 years ago

In the function _sanitize_to_target_branch there is a return statement as follow

return "Successfully sanitized repo" + (
    f" to pull request branch\n\nrun 'git switch "
    f"{effective_target_branch}' to checkout the branch"
    if self.create_pr_branch else ""
)

I had accidentaly removed the if statement and the tests still passed, meaning there is no check for if the text contains "pull request" and so on when we run with the create_pr_branch flag.

If codecov doesnt complain this is probably not worth testing

slarse commented 3 years ago

If codecov doesnt complain this is probably not worth testing

Codecov says nothing about test quality. It only tells you if a test traverses a given branch or statement. You can have 100% coverage and 0 assertions. I.e. you can have 100% coverage yet no meaningful tests.

In short: code coverage can only ever tell you what you're definitely not testing. It can't tell you if you're testing something in a meaningful way. Never rely on it for the latter.