nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.03k stars 435 forks source link

Pylint alerts corrections as part of an intervention experiment #1897

Open evidencebp opened 3 days ago

evidencebp commented 3 days ago

Is your feature request related to a problem? Please describe. I suhhest that I'll fix some Pylint alerts. Fixing the alelts should improve the code quality.

Describe the solution you'd like I'll fix the planed interventions

Describe alternatives you've considered Code can be kept as is.

Additional context

Pylint alerts are correlated with tendency to bugs and harder maintenance. I'd like to conduct a software engineering experiment regarding the benefit of Pylint alerts removal. The experiment is described here.

In the experiments, Pylint is used with some specific alerts, files are selected for intervention and control. After the interventions are done, one can wait and examine the results.

Your repository is expected to benefit from the interventions. I'm asking for your approval for conducting an intervention in your repository.

See examples of interventions in stanford-oval/storm, gabfl/vault, and coreruleset/coreruleset.

You can see the planed interventions

May I do the interventions, @nedbat ?

nedbat commented 3 days ago

Thanks for looking into this. I've adjusted my pylint settings to alert me to the kinds of problems I'm concerned about. I don't want to change the code simply because pylint suggests it. "Too many statements" is not enough of a reason to split functions into smaller parts, for example.

If you have specific improvements, and can give the reasons why you think they are better, I'm interested to see them.

I took a quick look at your experiment, but I didn't understand how you would determine if it was a success. If you change coverage.py code, how will you decide that it's better?

evidencebp commented 3 days ago

Part of the design of the experiment is to choose the intervention by random, to avoid selection bias. Therefore, during the experiment I cannot go over the alerts and apply only part. Of course, if the alert is a false alarm I will not intervene.

There are some local simple alerts that usually change a single line or so. Are you ok with the following? {'broad-exception-caught': 7, 'line-too-long': 4, 'superfluous-parens': 7, 'unnecessary-pass': 3, 'wildcard-import': 3}

The question of improvement is fundamental in software research and the various answers deal with many aspects. In the experiment, the first evaluation is using the developer's opinion. This is subjective and noisy but we will estimate that also I guess that most will agree that the influence of line-too-long is minor. A second line is code metrics (e.g., McCabe, lines of code). Last, there are process metrics (e.g., tendency to bugs, ease of modifications).

evidencebp commented 3 days ago

By the way, I noticed (using the using-constant-test alert), that there are places in the code with the pattern

If 0: Some code

Out of curiosity, what are their goals? Are these flags that are changed manually?