Rd4dev
commented
3 months ago Hey @BenHenning, just wanted to clear 2 main things up at the start.
1. MARKDOWN report format:
The .md report as I assume it to be most useful for CI runs, containing essential result data about the percentage of covered code, and can be added as a comment. This base template was discussed in our meeting notes.
Total coverage:
Files covered: (# changed / # run with coverage)
Coverage percentage: ##% covered / ##% expected
LOC: # covered / # instrumented
(indent left) Specific coverage:
app/src/.../app
home
HomeActivity.kt - 87% (110/115)
...
...The current Markdown report follows this foundational format and now appears as:
Coverage Report
Total coverage:
- Files covered: 1
- Covered File: utility/src/main/java/org/oppia/android/util/parser/math/MathModel.kt
- Coverage percentage: 87.50% covered
- Line coverage: 19 covered / 19 found
- Function coverage: 7 covered / 11 found
- Branch coverage: 2 covered / 2 found
Probably simplifying it to Line coverage: 19 / 19 covered, Function coverage: 7 / 11 covered, Branch 2 / 2 covered
I would like to know if any additional details that should be included in the report for local generation.
2. Path to Store Reports
Variable Path Definitions Across Different OS
Windows users typically use absolute paths like C:/Users/..., whereas in WSL, paths are like /mnt/c/Users.
So if we direct developers to use an absolute path, would this cause any confusion, and probably would it need to be handled in our file creation processes.
Providing and Handling Different Kinds of Paths
- Path Variants:
- When developers are directed to provide relative paths, some paths might be specified as
coverage/coverage.html, where the file extension could be incorrect (e.g., it might be a Markdown generation) - with commandbazel run //scripts:run_coverage -- $(pwd) utility/...parser/math/MathModel.kt MARKDOWN - Others might simply specify a directory like
coverage/, requiring us to check if a default filename should be provided.
- When developers are directed to provide relative paths, some paths might be specified as
Subpaths and Endless/Long Chains
-
Managing Subdirectories:
- For structured coverage reports, like for
utility/src/main/java/org/oppia/android/util/parser/math/MathModel.kt, I propose creating subdirectories under the same path to maintain organized coverage results. - Example structure:
(absolute/relative to $repoRoot)/utility/src/main/java/org/oppia/android/util/parser/math/MathModel/coverage.html. - I took this approach just to ensure the coverage reports are structured and possible re-runs of other files don't overwrite the same file, which would otherwise complicate referencing other file coverages (when needed).
- For structured coverage reports, like for
-
Handling Long Input Paths:
- But with this above approach if we consider scenarios where users provide long paths themselves (
bazel run ... /coverageData/report/path/place/here/) our mapping would result in $repoRoot/(user_defined_path)coverageData/report/path/place/here/(our_default_path)/utility/src/main/java/org/oppia/android/util/parser/math/MathModel/coverage.html.
- But with this above approach if we consider scenarios where users provide long paths themselves (
I just don't know, if this is a simple process and its just something that I complicate thinking of weird possibilities and approaches
Just stuck to something simple on both ends for now
I've implemented a solution based on these considerations, but it deviates from our previous discussions. I'm seeking your feedback to refine and adjust as necessary.
So basically, I just made it simple to not get developers input itself and make use of the filePath argument and create a dir coverage_reports in the repoRoot, based on the HTML or MARKDOWN arg
so for a command -
bazel run //scripts:run_coverage -- $(pwd) utility/src/main/java/org/oppia/android/util/parser/math/MathModel.kt HTML
// it would be
reportOutputPath: oppia-android/coverage_reports/utility/src/main/java/org/oppia/android/util/parser/math/MathModel/coverage.htmland for
bazel run //scripts:run_coverage -- $(pwd) app/src/main/java/org/oppia/android/app/mydownloads/MyDownloadsActivity.kt MARKDOWN
// it would be
reportOutputPath: oppia-android/coverage_reports/app/src/main/java/org/oppia/android/app/mydownloads/MyDownloadsActivity/coverage.md(Even this feels like a train)
- Version Control Consideration:
- To prevent unintentional inclusion in the GitHub repository, the coverage_reports dir is added to
.gitignore.
- To prevent unintentional inclusion in the GitHub repository, the coverage_reports dir is added to
Would it be possible to clear this up, if we can just proceed this way or should taking inputs from developers/users be mandatory?
github-actions[bot]
BenHenning
adhiamboperes
oppiabot[bot]
Explanation
Fixes part of #5343
Project
[PR 1.5 of Project 4.1]
Changes Made
This PR introduces the
CoverageReporterutility to generate the code coverage report. The main features and changes include:New Utility:
CoverageReporterto generate code coverage reports.Command Line Arguments:
HTMLorMARKDOWN.Report Generation:
coverage_reportsfolder relative to therepoRoot.Gitignore Update:
coverage_reportsfolder is added to.gitignoreto ensure it is not added to the repository.Essential Checklist
For UI-specific PRs only
If your PR includes UI-related changes, then: