search

ansible-collections / community.hashi_vault

Ansible collection for managing and working with HashiCorp Vault.
https://docs.ansible.com/ansible/devel/collections/community/hashi_vault/index.html
GNU General Public License v3.0
80 stars 58 forks source link

make local integration faster #423

Closed briantist closed 6 months ago

briantist commented 6 months ago
SUMMARY

The local integration runs are the longest in the CI, since they run the full integration suite 3 times. The idea is to ensure that integration tests still work properly in multiple runs against the same running Vault containers. Many of the integration tests have setup/teardown code that by our convention, must be idempotent. Many of them also use an optimization whereby they write a canary value to a cubbyhole, and if that value exists, the setup code is skipped, which saves a little bit of time.

So in CI, we run a normal "first run", a second run against the same containers (should skip setup code where canary values are found), and then a third run where we delete the cubbyhole canary values to ensure that the setup code is actually idempotent regardless of the canary values.

We run the LI tests against devel and the latest stable version of core, so we have two of these runs.

This PR attempts to let us optimize this process a little: one of the core versions will test with canary deletion, and the other won't, so each of those jobs will run the full integration suite twice instead of thrice, and we still cover both scenarios.

ISSUE TYPE
codecov[bot] commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (08a3809) 98.83% compared to head (d88ded7) 98.83%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #423 +/- ## ======================================= Coverage 98.83% 98.83% ======================================= Files 79 79 Lines 4197 4197 Branches 825 825 ======================================= Hits 4148 4148 Misses 39 39 Partials 10 10 ``` | [Flag](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | Coverage Δ | | |---|---|---| | [env_docker-default](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `98.83% <ø> (ø)` | | | [integration](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `81.25% <ø> (ø)` | | | [sanity](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `39.24% <ø> (ø)` | | | [target_auth_approle](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `89.47% <ø> (ø)` | | | [target_auth_aws_iam](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `50.00% <ø> (ø)` | | | [target_auth_azure](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `53.84% <ø> (ø)` | | | [target_auth_cert](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `86.36% <ø> (ø)` | | | [target_auth_jwt](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `91.30% <ø> (ø)` | | | [target_auth_ldap](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `89.47% <ø> (ø)` | | | [target_auth_none](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `100.00% <ø> (ø)` | | | [target_auth_token](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `71.42% <ø> (ø)` | | | [target_auth_userpass](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `85.71% <ø> (ø)` | | | [target_connection_options](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `74.76% <ø> (ø)` | | | [target_controller](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `83.40% <ø> (ø)` | | | [target_filter_vault_login_token](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `77.77% <ø> (ø)` | | | [target_import](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `39.24% <ø> (ø)` | | | [target_lookup_hashi_vault](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `81.33% <ø> (ø)` | | | [target_lookup_vault_ansible_settings](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `56.00% <ø> (ø)` | | | [target_lookup_vault_kv1_get](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `91.30% <ø> (ø)` | | | [target_lookup_vault_kv2_get](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `91.11% <ø> (ø)` | | | [target_lookup_vault_list](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `90.00% <ø> (ø)` | | | [target_lookup_vault_login](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `88.57% <ø> (ø)` | | | [target_lookup_vault_read](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `90.00% <ø> (ø)` | | | [target_lookup_vault_token_create](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `79.24% <ø> (ø)` | | | [target_lookup_vault_write](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `56.95% <ø> (ø)` | | | [target_module_utils](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `96.74% <ø> (ø)` | | | [target_module_vault_kv1_get](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `87.50% <ø> (ø)` | | | [target_module_vault_kv2_delete](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `56.85% <ø> (ø)` | | | [target_module_vault_kv2_get](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `87.23% <ø> (ø)` | | | [target_module_vault_kv2_write](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `57.26% <ø> (ø)` | | | [target_module_vault_list](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `85.71% <ø> (ø)` | | | [target_module_vault_login](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `83.72% <ø> (ø)` | | | [target_module_vault_pki_generate_certificate](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `78.72% <ø> (ø)` | | | [target_module_vault_read](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `85.71% <ø> (ø)` | | | [target_module_vault_token_create](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `91.66% <ø> (ø)` | | | [target_module_vault_write](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `56.02% <ø> (ø)` | | | [target_modules](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `82.67% <ø> (ø)` | | | [units](https://app.codecov.io/gh/ansible-collections/community.hashi_vault/pull/423/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections) | `95.49% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ansible-collections#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.