es loader error after R20220323-P6 software upgrade

[stevec1980]

Hello,

We are receiving the following error in the cloudwatch logs for the es loader since the software on our cluster was upgraded to R20220323-P6.

[ERROR] AuthenticationException: AuthenticationException(401, '')Traceback (most recent call last):  File "/var/task/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate

Is this a know issue?

[nakajiak]

https://github.com/aws-samples/siem-on-amazon-opensearch-service/issues/297

[nakajiak]

Hi @stevec1980, thanks for the feedback. This bug seems to be caused by some changes to sivg4 inside the OpenSearch service. We made the patch and have released v2.8.0c now. Please upgrade SIEM to v2.8.0c.

or manually change http_compress=True to http_compress=False in siem/utils.py from Lambda function es-loader https://github.com/aws-samples/siem-on-amazon-opensearch-service/blob/1083120d/source/lambda/es_loader/siem/utils.py#L324

then if you use v2.8.0 or v2.8.0c, please read this doc and re-ingest logs to opensearch https://github.com/aws-samples/siem-on-amazon-opensearch-service/blob/main/docs/configure_siem.md#loading-data-from-dead-sqs-dead-letter-queur

[stevec1980]

Thank you, this has fixed the issue.

Interestingly, it was working fine on Opensearch 1.3 for a couple of weeks but when the software was upgraded to P6 it broke.

Not sure if that is useful info for you or not?

Thanks again.

[nakajiak]

I have checked the initial installation in the following regions, but at the moment I can no longer reproduce the error.

version	region	status
OpenSearch v1.3 R20220323-P6	us-east-1, us-east-2, ap-northeast-1	no error
OpenSearch v1.3 R20220928	us-west-1, us-west-2, eu-central-1, eu-west-1, eu-west-2, ap-southeast-1	no error
OpenSearch v1.3 R20220928-P1	eu-west-3	no error

If an error occurs, wait for the release of R20220928 and apply it, or apply v2.8.0c or later of SIEM on OpanSearch.