codecov[bot]
commented
3 years ago Codecov Report
Merging #142 (76b760a) into main (ac2f9a5) will not change coverage. The diff coverage is
n/a.:exclamation: Current head 76b760a differs from pull request most recent head bb2b194. Consider uploading reports for the commit bb2b194 to get more accurate results
@@ Coverage Diff @@
## main #142 +/- ##
=======================================
Coverage 89.22% 89.22%
=======================================
Files 34 34
Lines 1346 1346
Branches 102 102
=======================================
Hits 1201 1201
Misses 135 135
Partials 10 10 | Flag | Coverage Δ | |
|---|---|---|
| env_docker-default | 89.22% <ø> (ø) |
|
| integration | 71.13% <ø> (ø) |
|
| py2.6 | 34.96% <ø> (ø) |
|
| py2.7 | 80.75% <ø> (ø) |
|
| py3.10 | 88.33% <ø> (ø) |
|
| py3.5 | 81.05% <ø> (ø) |
|
| py3.6 | 81.05% <ø> (ø) |
|
| py3.7 | 81.05% <ø> (ø) |
|
| py3.8 | 88.33% <ø> (ø) |
|
| py3.9 | 88.33% <ø> (ø) |
|
| sanity | 34.95% <ø> (ø) |
|
| target_ansible-doc | 36.16% <ø> (ø) |
|
| target_auth_approle | 84.21% <ø> (ø) |
|
| target_auth_jwt | 86.95% <ø> (ø) |
|
| target_auth_none | 100.00% <ø> (ø) |
|
| target_auth_token | 73.33% <ø> (ø) |
|
| target_connection_options | 73.78% <ø> (ø) |
|
| target_controller | 67.67% <ø> (ø) |
|
| target_import | 34.26% <ø> (ø) |
|
| target_lookup_hashi_vault | 78.75% <ø> (ø) |
|
| target_module_utils | 87.92% <ø> (ø) |
|
| units | 86.62% <ø> (ø) |
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update ac2f9a5...bb2b194. Read the comment docs.
github-actions[bot]
SUMMARY
Updating the version of Vault to test against has been a manual process. I'm trying to move toward the pattern of integration tests defaulting to the latest non-prelease Version of Vault (single version) when run locally.
For CI, I'll continue testing the latest two minor versions, as described in the README (and clarified further in this PR). That's now automatic, and finding the versions is done through a custom GitHub action via a Python script, which is configurable so we can easily change which and how many versions of Vault to test against.
So the integration tests now run twice in each job, rather than being matrixed with Vault version. It removes a little bit of parallelism but I think that's a benefit because the limit of active jobs in GHA was already limiting. At least within the same job, the Ansible test images have already been pulled, so we're saving some time on initialization. Setting the up a different Vault test container only takes about 10s.
This should have resulted in much better overall CI time, but I continue to be plagued by the fact that the 2.9 integration tests always run last, and they take the longest (the test images are much bigger, and take longer to pull, and 2.9 just plain runs slower in general).
I did experiment with changing the order in the matrix to see if I could get GitHub to run them in a different order, but it doesn't work. They display in a different order, but run in the same order, so 🤷♂️.
ISSUE TYPE
COMPONENT NAME
tests/integration
ADDITIONAL INFORMATION
N/A