jertel / elastalert2

ElastAlert 2 is a continuation of the original yelp/elastalert project. Pull requests are appreciated!
https://elastalert2.readthedocs.org
Apache License 2.0
895 stars 282 forks source link

Option skip_invalid not working for schema validation #1336

Closed jertel closed 9 months ago

jertel commented 9 months ago

Discussed in https://github.com/jertel/elastalert2/discussions/1333

Originally posted by **missaeldenadai** December 4, 2023 Hi, If a rule file is invalid due to schema validation, it is not being skipped and hence our ElastAlert2 is not deploying. My rule containing the schema error (the error is on `metric_agg_type: percentile`): ``` payment_send_spike_latency: |- --- name: some_name type: spike_aggregation index: index_name filter: - query: some_query timeframe: hours: 1 metric_agg_key: transaction.duration.us metric_agg_type: percentile percentile_range: 95 spike_type: "up" spike_height: 1.5 min_doc_count: 10 alert: opsgenie realert: minutes: 60 ``` The stack trace: ``` Index elastalert already exists. Skipping index creation. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/elastalert/loaders.py", line 299, in load_options self.rule_schema.validate(rule) File "/usr/local/lib/python3.11/site-packages/jsonschema/validators.py", line 430, in validate raise error jsonschema.exceptions.ValidationError: , 'rule_file': '/opt/elastalert/rules/payment_send_spike_latency.yaml'} is not valid under any of the given schemas Failed validating 'oneOf' in schema: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/elastalert", line 8, in sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 1868, in main client = ElastAlerter(args) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 127, in __init__ self.rules = self.rules_loader.load(self.conf, self.args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/elastalert/loaders.py", line 176, in load raise EAException('Error loading file %s: %s' % (rule_file, e)) elastalert.util.EAException: Error loading file /opt/elastalert/rules/payment_send_spike_latency.yaml: Invalid Rule file: /opt/elastalert/rules/payment_send_spike_latency.yaml is not valid under any of the given schemas Failed validating 'oneOf' in schema: ``` Does `skip_invalid` work for schema validation?