opensearch-project / opensearch-py

Python Client for OpenSearch
https://opensearch.org/docs/latest/clients/python/
Apache License 2.0
338 stars 170 forks source link

[BUG] AWSV4SignerAuth does not accept "aos" #534

Closed IanHoang closed 11 months ago

IanHoang commented 11 months ago

What is the bug?

When users are using OSB to test a SigV4 authenticated opensearch cluster, they set the environment variable to aos or aoss. However, AWSSigV4Auth class takes es instead of aos. This could be misleading to new OpenSearch users.

Are there any plans to support aos on top of es and aoss?

How can one reproduce the bug?

  1. Create an AOS domain that uses an authenticated IAM user for security
  2. Set OSB environment variables as described here
    OSB_AWS_ACCESS_KEY_ID=<IAM USER AWS ACCESS KEY ID>
    OSB_AWS_SECRET_ACCESS_KEY=<IAM USER AWS SECRET ACCESS KEY>
    OSB_REGION=<YOUR REGION>
    OSB_SERVICE=aos
  3. Run a test like the following command:
    opensearch-benchmark execute-test \
    --target-hosts=<CLUSTER ENDPOINT> \
    --pipeline=benchmark-only \
    --workload=geonames \
    --client-options=timeout:120,amazon_aws_log_in:environment \

Error encountered:

"Service for aws log in should be one of 'es' or 'aoss'"

What is the expected behavior?

opensearch-py should accept aos as well on top of es and aoss.

What is your host/environment?

Occurs on MacOS, Linux and with Amazon Managed-Service Clusters

Do you have any screenshots?

N/A

Do you have any additional context?

N/A

dblock commented 11 months ago

This feels like an OSB problem because the AWS service is definitely es and aoss, not aos, no?

IanHoang commented 11 months ago

@dblock AWS service uses es for AWS CLI still but OSB only has a check added by a contributor a few months back:

        if aws_log_in_dict["service"] not in ['es', 'aoss']:
            self.logger.error("Service for aws log in should be one of 'es' or 'aoss'")
            raise exceptions.SystemSetupError(
                "Cannot specify service as '{}'. Accepted values are 'es' or 'aoss'.".format(
                    aws_log_in_dict["service"])
            )

Since OSB relies on opensearch-py to communicate with target clusters, I believe the contributor added this check because opensearch-py might not support aos at the moment and also has the following line: https://github.com/opensearch-project/opensearch-py/blob/84ac172ddc54b3e6c975d36221d16ec3e78a2fe9/opensearchpy/helpers/signer.py#L51

dblock commented 11 months ago

@IanHoang That default is correct, the service code is "es". Service codes are used to scope service credentials. Do you not agree?

IanHoang commented 11 months ago

@dblock You're right, thanks for calling this out! opensearch-py uses botocore, which still uses es. Closing this issue