Create build plugin pipeline

[mashhurs]

Description

Replacing #93 since repo went public, I had to re-open with a new PR.

A plugin build pipeline which builds the plugin on various ES versions including snapshots and main branch to detect the issues earlies.

Tasks considered:

• [x] Build on latest release (dynamically resolve 8.11) and run integration tests with release version (same 8.11) docker
• [x] Build on latest snapshot (dynamically resolve 8.11.x-snapshot) and run integration tests with snapshot version (same 8.11.x-snapshot) docker
  • possible after landing https://github.com/elastic/logstash/pull/15631
• [x] Build on main and run integration tests with main snapshot version (8.12 for this case) docker
• [x] A definition to send slack notification when fails

How to test locally?

• NOTE: CI is 🔴 because pipeline names renamed accordingly. This commit temporarily renames the pipeline to make sure changes are 🟢

• Run ./.ci/docker-setup.sh & ./.ci/docker-run.sh to test on local

• Placing local integration test logs here

  ELASTIC_STACK_VERSION=8.11.0, SNAPSHOT=false

  ci-logstash-1  | LogStash::Filters::ElasticIntegration
  ci-logstash-1  |   the plugin class
  ci-logstash-1  |     is expected to have attributes {:config_name => "elastic_integration"}
  ci-logstash-1  |     is expected to be <= LogStash::Filters::Base
  ci-logstash-1  |     is expected to be a kind of Class
  ci-logstash-1  |     on unsupported Java
  ci-logstash-1  |       prevents initialization and presents helpful guidancee
  ci-logstash-1  |   Logstash core requirements
  ci-logstash-1  |     on OSS-only Logstash
  ci-logstash-1  |       cannot be instantiated
  ci-logstash-1  |     on complete Logstash
  ci-logstash-1  |       can be instantiated
  ci-logstash-1  |   an instance with default config
  ci-logstash-1  |     is expected to respond to #filter with 1 argument
  ci-logstash-1  |     is expected to respond to #register with 0 arguments
  ci-logstash-1  |     is expected to be a kind of LogStash::Filters::Base
  ci-logstash-1  |   plugin register
  ci-logstash-1  |     with SSL disabled
  ci-logstash-1  |       with `cloud_id`
  ci-logstash-1  |         raises an error
  ci-logstash-1  |       with SSL related configs
  ci-logstash-1  |         does not allow and raises an error
  ci-logstash-1  |       connection prerequisites
  ci-logstash-1  |         basic auth
  ci-logstash-1  |           when `password` is an empty
  ci-logstash-1  |             requires non-empty `username`
  ci-logstash-1  |           with `password`
  ci-logstash-1  |             requires `username`
  ci-logstash-1  |           when `username` is an empty
  ci-logstash-1  |             requires non-empty `username`
  ci-logstash-1  |           with `username`
  ci-logstash-1  |             requires `password`
  ci-logstash-1  |         with multiple auth options
  ci-logstash-1  |           does not allow
  ci-logstash-1  |         with no `hosts` or `cloud_id
  ci-logstash-1  |           requires either of them
  ci-logstash-1  |         `hosts`
  ci-logstash-1  |           with multiple hosts
  ci-logstash-1  |             applies default value
  ci-logstash-1  |           with HTTP scheme
  ci-logstash-1  |             accepts
  ci-logstash-1  |           single
  ci-logstash-1  |             applies default values
  ci-logstash-1  |           with HTTPS scheme
  ci-logstash-1  |             enforces to agree with scheme
  ci-logstash-1  |         with empty auth option
  ci-logstash-1  |           does not allow
  ci-logstash-1  |         with either `hosts` or `cloud_id
  ci-logstash-1  |           accepts
  ci-logstash-1  |       with otherwise minimal config
  ci-logstash-1  |         does not have a value for meaningless ssl-related settings
  ci-logstash-1  |     connection prerequisites
  ci-logstash-1  |       with both `hosts` and `cloud_id`
  ci-logstash-1  |         raises an error
  ci-logstash-1  |       `hosts`
  ci-logstash-1  |         HTTP scheme
  ci-logstash-1  |           with SSL disabled
  ci-logstash-1  |             accepts
  ci-logstash-1  |           with SSL enabled
  ci-logstash-1  |             enforces to use HTTPS
  ci-logstash-1  |         with HTTPS scheme
  ci-logstash-1  |           when SSL enabled
  ci-logstash-1  |             accepts
  ci-logstash-1  |           when SSL disabled
  ci-logstash-1  |             raises an error
  ci-logstash-1  |     normalize `hosts`
  ci-logstash-1  |       with SSL enabled
  ci-logstash-1  |         applies default value
  ci-logstash-1  |       with SSL disabled
  ci-logstash-1  |         applies default value
  ci-logstash-1  |       with no SSL specified
  ci-logstash-1  |         has ssl_verification_mode => full
  ci-logstash-1  |         enables SSL
  ci-logstash-1  |         applies default value
  ci-logstash-1  |     infer SSL from connection settings
  ci-logstash-1  |       with `cloud_id`
  ci-logstash-1  |         has ssl_verification_mode => full
  ci-logstash-1  |         enables SSL
  ci-logstash-1  |       `hosts`
  ci-logstash-1  |         with HTTPS protocol entry
  ci-logstash-1  |           enables SSL
  ci-logstash-1  |           has ssl_verification_mode => full
  ci-logstash-1  |         with empty entry protocol
  ci-logstash-1  |           enables SSL
  ci-logstash-1  |           has ssl_verification_mode => full
  ci-logstash-1  |         with HTTP protocol entry
  ci-logstash-1  |           disables the SSL
  ci-logstash-1  |         with multiple paths
  ci-logstash-1  |           raises an error
  ci-logstash-1  |         with mixed entries protocol
  ci-logstash-1  |           raises an error
  ci-logstash-1  |     with SSL enabled
  ci-logstash-1  |       has ssl_verification_mode => full
  ci-logstash-1  |       with `ssl_verification_mode` is `none`
  ci-logstash-1  |         with `ssl_truststore_path`
  ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
  ci-logstash-1  |         with `ssl_certificate_authorities`
  ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
  ci-logstash-1  |         with `ssl_truststore_password`
  ci-logstash-1  |           requires `ssl_truststore_password` and full or certificate `ssl_verification_mode`
  ci-logstash-1  |       without `ssl_certificate`
  ci-logstash-1  |         without `ssl_key`
  ci-logstash-1  |           with `ssl_key_passphrase`
  ci-logstash-1  |             with `ssl_key_passphrase`
  ci-logstash-1  |               requires `ssl_key`
  ci-logstash-1  |         with `ssl_key`
  ci-logstash-1  |           requires `ssl_certificate`
  ci-logstash-1  |       with `ssl_keystore_path`
  ci-logstash-1  |         requires `ssl_keystore_password`
  ci-logstash-1  |         with non-empty `ssl_keystore_password`
  ci-logstash-1  |           with readable and writable path
  ci-logstash-1  |             requires non-writable path
  ci-logstash-1  |           with non readable path
  ci-logstash-1  |             requires readable path
  ci-logstash-1  |         with empty `ssl_keystore_password`
  ci-logstash-1  |           doesn't allow empty `ssl_keystore_password`
  ci-logstash-1  |       with `ssl_verification_mode` is not `none`
  ci-logstash-1  |         with `ssl_truststore_password`
  ci-logstash-1  |           requires `ssl_truststore_path`
  ci-logstash-1  |         with `ssl_truststore_path` and `ssl_certificate_authorities`
  ci-logstash-1  |           doesn't allow to use together
  ci-logstash-1  |         with `ssl_certificate_authorities`
  ci-logstash-1  |           with readable and writable path
  ci-logstash-1  |             requires non-writable path
  ci-logstash-1  |           with non readable path
  ci-logstash-1  |             requires readable path
  ci-logstash-1  |         with `ssl_truststore_path`
  ci-logstash-1  |           requires `ssl_truststore_password`
  ci-logstash-1  |           with empty `ssl_truststore_password`
  ci-logstash-1  |             doesn't accept empty `ssl_truststore_password`
  ci-logstash-1  |           with non-empty `ssl_truststore_password`
  ci-logstash-1  |             with readable and writable path
  ci-logstash-1  |               requires non-writable path
  ci-logstash-1  |             with non readable path
  ci-logstash-1  |               requires readable path
  ci-logstash-1  |         without `ssl_truststore_path` and empty `ssl_certificate_authorities`
  ci-logstash-1  |           doesn't allow empty `ssl_certificate_authorities`
  ci-logstash-1  |       connection prerequisites
  ci-logstash-1  |         with both `hosts` and `cloud_id`
  ci-logstash-1  |           raises an error
  ci-logstash-1  |         with `hosts`
  ci-logstash-1  |           with HTTPS scheme
  ci-logstash-1  |             accepts
  ci-logstash-1  |           with multiple hosts
  ci-logstash-1  |             applies default value
  ci-logstash-1  |           with HTTP scheme
  ci-logstash-1  |             enforces to agree with scheme
  ci-logstash-1  |       with `ssl_certificate`
  ci-logstash-1  |         requires `ssl_key`
  ci-logstash-1  |         with `ssl_keystore_path`
  ci-logstash-1  |           doesn't allow to use together
  ci-logstash-1  |         with `ssl_key`
  ci-logstash-1  |           requires `ssl_key_passphrase`
  ci-logstash-1  |           with empty `ssl_key_passphrase`
  ci-logstash-1  |             requires non-empty passphrase
  ci-logstash-1  |           with non-empty `ssl_key_passphrase`
  ci-logstash-1  |             with readable and writable path
  ci-logstash-1  |               requires non-writable path
  ci-logstash-1  |             with non readable path
  ci-logstash-1  |               requires readable path
  ci-logstash-1  |       without `ssl_keystore_path`
  ci-logstash-1  |         with `ssl_keystore_password`
  ci-logstash-1  |           doesn't allow
  ci-logstash-1  | 
  ci-logstash-1  | Finished in 3.79 seconds (files took 3.8 seconds to load)
  ci-logstash-1  | 73 examples, 0 failures

ELASTIC_STACK_VERSION=8.x & SNAPSHOT=true, translates to 8.12.0-SNAPSHOT, equals to main branch

ci-logstash-1  | LogStash::Filters::ElasticIntegration
ci-logstash-1  |   plugin register
ci-logstash-1  |     with SSL disabled
ci-logstash-1  |       with SSL related configs
ci-logstash-1  |         does not allow and raises an error
ci-logstash-1  |       with otherwise minimal config
ci-logstash-1  |         does not have a value for meaningless ssl-related settings
ci-logstash-1  |       connection prerequisites
ci-logstash-1  |         with either `hosts` or `cloud_id
ci-logstash-1  |           accepts
ci-logstash-1  |         with no `hosts` or `cloud_id
ci-logstash-1  |           requires either of them
ci-logstash-1  |         `hosts`
ci-logstash-1  |           with multiple hosts
ci-logstash-1  |             applies default value
ci-logstash-1  |           with HTTPS scheme
ci-logstash-1  |             enforces to agree with scheme
ci-logstash-1  |           with HTTP scheme
ci-logstash-1  |             accepts
ci-logstash-1  |           single
ci-logstash-1  |             applies default values
ci-logstash-1  |         with multiple auth options
ci-logstash-1  |           does not allow
ci-logstash-1  |         basic auth
ci-logstash-1  |           with `username`
ci-logstash-1  |             requires `password`
ci-logstash-1  |           when `username` is an empty
ci-logstash-1  |             requires non-empty `username`
ci-logstash-1  |           when `password` is an empty
ci-logstash-1  |             requires non-empty `username`
ci-logstash-1  |           with `password`
ci-logstash-1  |             requires `username`
ci-logstash-1  |         with empty auth option
ci-logstash-1  |           does not allow
ci-logstash-1  |       with `cloud_id`
ci-logstash-1  |         raises an error
ci-logstash-1  |     with SSL enabled
ci-logstash-1  |       has ssl_verification_mode => full
ci-logstash-1  |       without `ssl_keystore_path`
ci-logstash-1  |         with `ssl_keystore_password`
ci-logstash-1  |           doesn't allow
ci-logstash-1  |       connection prerequisites
ci-logstash-1  |         with both `hosts` and `cloud_id`
ci-logstash-1  |           raises an error
ci-logstash-1  |         with `hosts`
ci-logstash-1  |           with HTTP scheme
ci-logstash-1  |             enforces to agree with scheme
ci-logstash-1  |           with multiple hosts
ci-logstash-1  |             applies default value
ci-logstash-1  |           with HTTPS scheme
ci-logstash-1  |             accepts
ci-logstash-1  |       without `ssl_certificate`
ci-logstash-1  |         with `ssl_key`
ci-logstash-1  |           requires `ssl_certificate`
ci-logstash-1  |         without `ssl_key`
ci-logstash-1  |           with `ssl_key_passphrase`
ci-logstash-1  |             with `ssl_key_passphrase`
ci-logstash-1  |               requires `ssl_key`
ci-logstash-1  |       with `ssl_certificate`
ci-logstash-1  |         requires `ssl_key`
ci-logstash-1  |         with `ssl_keystore_path`
ci-logstash-1  |           doesn't allow to use together
ci-logstash-1  |         with `ssl_key`
ci-logstash-1  |           requires `ssl_key_passphrase`
ci-logstash-1  |           with empty `ssl_key_passphrase`
ci-logstash-1  |             requires non-empty passphrase
ci-logstash-1  |           with non-empty `ssl_key_passphrase`
ci-logstash-1  |             with non readable path
ci-logstash-1  |               requires readable path
ci-logstash-1  |             with readable and writable path
ci-logstash-1  |               requires non-writable path
ci-logstash-1  |       with `ssl_verification_mode` is `none`
ci-logstash-1  |         with `ssl_truststore_path`
ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
ci-logstash-1  |         with `ssl_truststore_password`
ci-logstash-1  |           requires `ssl_truststore_password` and full or certificate `ssl_verification_mode`
ci-logstash-1  |         with `ssl_certificate_authorities`
ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
ci-logstash-1  |       with `ssl_verification_mode` is not `none`
ci-logstash-1  |         with `ssl_truststore_password`
ci-logstash-1  |           requires `ssl_truststore_path`
ci-logstash-1  |         with `ssl_truststore_path` and `ssl_certificate_authorities`
ci-logstash-1  |           doesn't allow to use together
ci-logstash-1  |         with `ssl_certificate_authorities`
ci-logstash-1  |           with non readable path
ci-logstash-1  |             requires readable path
ci-logstash-1  |           with readable and writable path
ci-logstash-1  |             requires non-writable path
ci-logstash-1  |         without `ssl_truststore_path` and empty `ssl_certificate_authorities`
ci-logstash-1  |           doesn't allow empty `ssl_certificate_authorities`
ci-logstash-1  |         with `ssl_truststore_path`
ci-logstash-1  |           requires `ssl_truststore_password`
ci-logstash-1  |           with empty `ssl_truststore_password`
ci-logstash-1  |             doesn't accept empty `ssl_truststore_password`
ci-logstash-1  |           with non-empty `ssl_truststore_password`
ci-logstash-1  |             with readable and writable path
ci-logstash-1  |               requires non-writable path
ci-logstash-1  |             with non readable path
ci-logstash-1  |               requires readable path
ci-logstash-1  |       with `ssl_keystore_path`
ci-logstash-1  |         requires `ssl_keystore_password`
ci-logstash-1  |         with empty `ssl_keystore_password`
ci-logstash-1  |           doesn't allow empty `ssl_keystore_password`
ci-logstash-1  |         with non-empty `ssl_keystore_password`
ci-logstash-1  |           with readable and writable path
ci-logstash-1  |             requires non-writable path
ci-logstash-1  |           with non readable path
ci-logstash-1  |             requires readable path
ci-logstash-1  |     normalize `hosts`
ci-logstash-1  |       with SSL disabled
ci-logstash-1  |         applies default value
ci-logstash-1  |       with no SSL specified
ci-logstash-1  |         has ssl_verification_mode => full
ci-logstash-1  |         enables SSL
ci-logstash-1  |         applies default value
ci-logstash-1  |       with SSL enabled
ci-logstash-1  |         applies default value
ci-logstash-1  |     connection prerequisites
ci-logstash-1  |       with both `hosts` and `cloud_id`
ci-logstash-1  |         raises an error
ci-logstash-1  |       `hosts`
ci-logstash-1  |         HTTP scheme
ci-logstash-1  |           with SSL enabled
ci-logstash-1  |             enforces to use HTTPS
ci-logstash-1  |           with SSL disabled
ci-logstash-1  |             accepts
ci-logstash-1  |         with HTTPS scheme
ci-logstash-1  |           when SSL disabled
ci-logstash-1  |             raises an error
ci-logstash-1  |           when SSL enabled
ci-logstash-1  |             accepts
ci-logstash-1  |     infer SSL from connection settings
ci-logstash-1  |       `hosts`
ci-logstash-1  |         with mixed entries protocol
ci-logstash-1  |           raises an error
ci-logstash-1  |         with HTTP protocol entry
ci-logstash-1  |           disables the SSL
ci-logstash-1  |         with HTTPS protocol entry
ci-logstash-1  |           has ssl_verification_mode => full
ci-logstash-1  |           enables SSL
ci-logstash-1  |         with empty entry protocol
ci-logstash-1  |           enables SSL
ci-logstash-1  |           has ssl_verification_mode => full
ci-logstash-1  |         with multiple paths
ci-logstash-1  |           raises an error
ci-logstash-1  |       with `cloud_id`
ci-logstash-1  |         enables SSL
ci-logstash-1  |         has ssl_verification_mode => full
ci-logstash-1  |   the plugin class
ci-logstash-1  |     is expected to have attributes {:config_name => "elastic_integration"}
ci-logstash-1  |     is expected to be <= LogStash::Filters::Base
ci-logstash-1  |     is expected to be a kind of Class
ci-logstash-1  |     on unsupported Java
ci-logstash-1  |       prevents initialization and presents helpful guidancee
ci-logstash-1  |   Logstash core requirements
ci-logstash-1  |     on complete Logstash
ci-logstash-1  |       can be instantiated
ci-logstash-1  |     on OSS-only Logstash
ci-logstash-1  |       cannot be instantiated
ci-logstash-1  |   an instance with default config
ci-logstash-1  |     is expected to respond to #filter with 1 argument
ci-logstash-1  |     is expected to be a kind of LogStash::Filters::Base
ci-logstash-1  |     is expected to respond to #register with 0 arguments
ci-logstash-1  | 
ci-logstash-1  | Finished in 3.84 seconds (files took 3.93 seconds to load)
ci-logstash-1  | 73 examples, 0 failures
ci-logstash-1  | 
ci-logstash-1  | Randomized with seed 33081

ELASTIC_STACK_VERSION=8.11.2-SNAPSHOT case

ci-logstash-1  | LogStash::Filters::ElasticIntegration
ci-logstash-1  |   an instance with default config
ci-logstash-1  |     is expected to respond to #register with 0 arguments
ci-logstash-1  |     is expected to respond to #filter with 1 argument
ci-logstash-1  |     is expected to be a kind of LogStash::Filters::Base
ci-logstash-1  |   plugin register
ci-logstash-1  |     normalize `hosts`
ci-logstash-1  |       with SSL disabled
ci-logstash-1  |         applies default value
ci-logstash-1  |       with SSL enabled
ci-logstash-1  |         applies default value
ci-logstash-1  |       with no SSL specified
ci-logstash-1  |         applies default value
ci-logstash-1  |         enables SSL
ci-logstash-1  |         has ssl_verification_mode => full
ci-logstash-1  |     connection prerequisites
ci-logstash-1  |       with both `hosts` and `cloud_id`
ci-logstash-1  |         raises an error
ci-logstash-1  |       `hosts`
ci-logstash-1  |         HTTP scheme
ci-logstash-1  |           with SSL enabled
ci-logstash-1  |             enforces to use HTTPS
ci-logstash-1  |           with SSL disabled
ci-logstash-1  |             accepts
ci-logstash-1  |         with HTTPS scheme
ci-logstash-1  |           when SSL enabled
ci-logstash-1  |             accepts
ci-logstash-1  |           when SSL disabled
ci-logstash-1  |             raises an error
ci-logstash-1  |     with SSL disabled
ci-logstash-1  |       connection prerequisites
ci-logstash-1  |         `hosts`
ci-logstash-1  |           with HTTP scheme
ci-logstash-1  |             accepts
ci-logstash-1  |           with HTTPS scheme
ci-logstash-1  |             enforces to agree with scheme
ci-logstash-1  |           single
ci-logstash-1  |             applies default values
ci-logstash-1  |           with multiple hosts
ci-logstash-1  |             applies default value
ci-logstash-1  |         basic auth
ci-logstash-1  |           with `password`
ci-logstash-1  |             requires `username`
ci-logstash-1  |           with `username`
ci-logstash-1  |             requires `password`
ci-logstash-1  |           when `username` is an empty
ci-logstash-1  |             requires non-empty `username`
ci-logstash-1  |           when `password` is an empty
ci-logstash-1  |             requires non-empty `username`
ci-logstash-1  |         with either `hosts` or `cloud_id
ci-logstash-1  |           accepts
ci-logstash-1  |         with multiple auth options
ci-logstash-1  |           does not allow
ci-logstash-1  |         with empty auth option
ci-logstash-1  |           does not allow
ci-logstash-1  |         with no `hosts` or `cloud_id
ci-logstash-1  |           requires either of them
ci-logstash-1  |       with `cloud_id`
ci-logstash-1  |         raises an error
ci-logstash-1  |       with SSL related configs
ci-logstash-1  |         does not allow and raises an error
ci-logstash-1  |       with otherwise minimal config
ci-logstash-1  |         does not have a value for meaningless ssl-related settings
ci-logstash-1  |     with SSL enabled
ci-logstash-1  |       has ssl_verification_mode => full
ci-logstash-1  |       connection prerequisites
ci-logstash-1  |         with `hosts`
ci-logstash-1  |           with HTTPS scheme
ci-logstash-1  |             accepts
ci-logstash-1  |           with multiple hosts
ci-logstash-1  |             applies default value
ci-logstash-1  |           with HTTP scheme
ci-logstash-1  |             enforces to agree with scheme
ci-logstash-1  |         with both `hosts` and `cloud_id`
ci-logstash-1  |           raises an error
ci-logstash-1  |       without `ssl_keystore_path`
ci-logstash-1  |         with `ssl_keystore_password`
ci-logstash-1  |           doesn't allow
ci-logstash-1  |       with `ssl_keystore_path`
ci-logstash-1  |         requires `ssl_keystore_password`
ci-logstash-1  |         with non-empty `ssl_keystore_password`
ci-logstash-1  |           with readable and writable path
ci-logstash-1  |             requires non-writable path
ci-logstash-1  |           with non readable path
ci-logstash-1  |             requires readable path
ci-logstash-1  |         with empty `ssl_keystore_password`
ci-logstash-1  |           doesn't allow empty `ssl_keystore_password`
ci-logstash-1  |       with `ssl_verification_mode` is not `none`
ci-logstash-1  |         with `ssl_truststore_path` and `ssl_certificate_authorities`
ci-logstash-1  |           doesn't allow to use together
ci-logstash-1  |         with `ssl_truststore_password`
ci-logstash-1  |           requires `ssl_truststore_path`
ci-logstash-1  |         without `ssl_truststore_path` and empty `ssl_certificate_authorities`
ci-logstash-1  |           doesn't allow empty `ssl_certificate_authorities`
ci-logstash-1  |         with `ssl_certificate_authorities`
ci-logstash-1  |           with readable and writable path
ci-logstash-1  |             requires non-writable path
ci-logstash-1  |           with non readable path
ci-logstash-1  |             requires readable path
ci-logstash-1  |         with `ssl_truststore_path`
ci-logstash-1  |           requires `ssl_truststore_password`
ci-logstash-1  |           with empty `ssl_truststore_password`
ci-logstash-1  |             doesn't accept empty `ssl_truststore_password`
ci-logstash-1  |           with non-empty `ssl_truststore_password`
ci-logstash-1  |             with non readable path
ci-logstash-1  |               requires readable path
ci-logstash-1  |             with readable and writable path
ci-logstash-1  |               requires non-writable path
ci-logstash-1  |       without `ssl_certificate`
ci-logstash-1  |         without `ssl_key`
ci-logstash-1  |           with `ssl_key_passphrase`
ci-logstash-1  |             with `ssl_key_passphrase`
ci-logstash-1  |               requires `ssl_key`
ci-logstash-1  |         with `ssl_key`
ci-logstash-1  |           requires `ssl_certificate`
ci-logstash-1  |       with `ssl_verification_mode` is `none`
ci-logstash-1  |         with `ssl_truststore_path`
ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
ci-logstash-1  |         with `ssl_truststore_password`
ci-logstash-1  |           requires `ssl_truststore_password` and full or certificate `ssl_verification_mode`
ci-logstash-1  |         with `ssl_certificate_authorities`
ci-logstash-1  |           requires full or certificate `ssl_verification_mode`
ci-logstash-1  |       with `ssl_certificate`
ci-logstash-1  |         requires `ssl_key`
ci-logstash-1  |         with `ssl_key`
ci-logstash-1  |           requires `ssl_key_passphrase`
ci-logstash-1  |           with empty `ssl_key_passphrase`
ci-logstash-1  |             requires non-empty passphrase
ci-logstash-1  |           with non-empty `ssl_key_passphrase`
ci-logstash-1  |             with readable and writable path
ci-logstash-1  |               requires non-writable path
ci-logstash-1  |             with non readable path
ci-logstash-1  |               requires readable path
ci-logstash-1  |         with `ssl_keystore_path`
ci-logstash-1  |           doesn't allow to use together
ci-logstash-1  |     infer SSL from connection settings
ci-logstash-1  |       `hosts`
ci-logstash-1  |         with empty entry protocol
ci-logstash-1  |           enables SSL
ci-logstash-1  |           has ssl_verification_mode => full
ci-logstash-1  |         with mixed entries protocol
ci-logstash-1  |           raises an error
ci-logstash-1  |         with multiple paths
ci-logstash-1  |           raises an error
ci-logstash-1  |         with HTTP protocol entry
ci-logstash-1  |           disables the SSL
ci-logstash-1  |         with HTTPS protocol entry
ci-logstash-1  |           has ssl_verification_mode => full
ci-logstash-1  |           enables SSL
ci-logstash-1  |       with `cloud_id`
ci-logstash-1  |         has ssl_verification_mode => full
ci-logstash-1  |         enables SSL
ci-logstash-1  |   the plugin class
ci-logstash-1  |     is expected to have attributes {:config_name => "elastic_integration"}
ci-logstash-1  |     is expected to be a kind of Class
ci-logstash-1  |     is expected to be <= LogStash::Filters::Base
ci-logstash-1  |     on unsupported Java
ci-logstash-1  |       prevents initialization and presents helpful guidancee
ci-logstash-1  |   Logstash core requirements
ci-logstash-1  |     on OSS-only Logstash
ci-logstash-1  |       cannot be instantiated
ci-logstash-1  |     on complete Logstash
ci-logstash-1  |       can be instantiated
ci-logstash-1  | 
ci-logstash-1  | Finished in 3.79 seconds (files took 3.72 seconds to load)
ci-logstash-1  | 73 examples, 0 failures

[mashhurs]

This is worth moving forward with, even if I have additional thoughts. Feel free to spin them off as a secondary feature-request.

Ideally, we would test daily against:

• the most recent release in the 8.x series (✅ maps to ELASTIC_STACK_VERSION=8.x SNAPSHOT=false)
• a snapshot of the next release in the 8.x series (❌ no current mapping, since logstash_releases.json has no entry representing 8.x that has artifacts built from 8.11, which as of this writing is where the next release will come from)
• the future release from main (🟡 currently indirectly satisfied by ELASTIC_STACK_VERSION=8.x SNAPSHOT=true, since we maintain outside knowledge that 8.12.0-SNAPSHOT is built from main)

With these changes, we get improvements, but I think we need to add an entry to the upstream logstash_releases.json to get what we are looking for.

I'm expressing in YAML because I need comments and clean diffs, but the source file of course is actually JSON.

As-implemented now, snapshots[8.x] points to an artifact built off of the main branch; once a feature branch is cut, we no longer point to it. snapshots[8.x] should be renamed to snapshots[main], and we should introduce a new snapshots[8.x] that points to a snapshot built off the latest 8.x branch:

 snapshots:
   "7.x": "7.17.16-SNAPSHOT" # next release in 7.x series
-  "8.x": "8.12.0-SNAPSHOT" # future minor (pre-ff)
+  "8.x": "8.11.2-SNAPSHOT" # next release in 8.x series
+  "main": "8.12.0-SNAPSHOT" # future minor (pre-ff)

When 8.12 feature-freezes, the diff of that FF would look like:

 snapshots:
   "7.x": "7.17.16-SNAPSHOT" # next release
-  "8.x": "8.11.2-SNAPSHOT"  # next release in 8.x series
+  "8.x": "8.12.0-SNAPSHOT"  # next release in 8.x series
-  "main": "8.12.0-SNAPSHOT" # future minor (pre-ff)
+  "main": "8.13.0-SNAPSHOT" # future minor (pre-ff)

When 8.11.2 drops the diff would look like:

 releases:
   "5.x": "5.6.16"
   "6.x": "6.8.23"
   "7.x": "7.17.15" # latest minor, latest patch
-  "8.x": "8.11.1" # latest minor, latest patch
+  "8.x": "8.11.2" # latest minor, latest patch
 snapshots:
   "7.x": "7.17.16-SNAPSHOT" # next release
-  "8.x": "8.11.2-SNAPSHOT"  # next release in 8.x series
+  "8.x": "8.11.3-SNAPSHOT"  # next release in 8.x series
   "main": "8.12.0-SNAPSHOT" # future minor (pre-ff)

When 8.12.0 drops the diff would look like:

 releases:
   "5.x": "5.6.16"
   "6.x": "6.8.23"
   "7.x": "7.17.15" # latest minor, latest patch
-  "8.x": "8.11.2" # latest minor, latest patch
+  "8.x": "8.12.0" # latest minor, latest patch
 snapshots:
   "7.x": "7.17.16-SNAPSHOT" # next release
-  "8.x": "8.11.3-SNAPSHOT"  # next release in 8.x series
+  "8.x": "8.12.1-SNAPSHOT"  # next release in 8.x series
   "main": "8.13.0-SNAPSHOT" # future minor (pre-ff)

This would give us the ability we need, to

• test against the latest release ELASTIC_STACK_VERSION=8.x SNAPSHOT=false
• test against the next release ELASTIC_STACK_VERSION=8.x SNAPSHOT=true
• test against upcoming changes pre-FF ELASTIC_STACK_VERSION=main SNAPSHOT=true

This is a wrap up from our offline discussion and the change to logstash_releases.json introduces main and updates next minor 8.11 release (current snapshot). Once sorted out, I am planning apply changes to our release process and share with the team. Note that, if we don't land the logstash_releases.json changes, current script ONLY covers current 8.11 release and main. If it lands, I will add 8.11 snapshot definition in the pipeline.

[mashhurs]

LGTM (provided the Buildkite job becomes green)

To provide more details, the pipeline becomes red actually and it is an expectation. The plugin is not synced with ES main branch yet, and it is planned.

[roaksoax]

Given that we need to create a build to test against 8.11, I think we should go ahead and land it as is. The failures on 8.12/main is because of the missing 8.12 ES snapshot. So lets land, and work next week to unblock this bui8ld on main/8.12