search

HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
19.58k stars 2.42k forks source link

EKS Pod Identity support (upgrade boto) #6124

Open zarnovican opened 4 months ago

zarnovican commented 4 months ago

Is your feature request related to a problem? Please describe.

With the existing boto/botocore you are getting stacktrace

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/generics.py", line 190, in post
    return self.create(request, *args, **kwargs)
  File "/label-studio/label_studio/./io_storages/api.py", line 163, in create
    serializer.is_valid(raise_exception=True)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/serializers.py", line 227, in is_valid
    self._validated_data = self.run_validation(self.initial_data)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/serializers.py", line 429, in run_validation
    value = self.validate(value)
  File "/label-studio/label_studio/./io_storages/s3/serializers.py", line 43, in validate
    storage.validate_connection()
  File "/label-studio/label_studio/./io_storages/s3/models.py", line 84, in validate_connection
    client = self.get_client()
  File "/label-studio/label_studio/./io_storages/s3/models.py", line 72, in get_client
    client, _ = self.get_client_and_resource()
  File "/label-studio/label_studio/./io_storages/s3/models.py", line 61, in get_client_and_resource
    result = get_client_and_resource(
  File "/label-studio/label_studio/./io_storages/s3/utils.py", line 38, in get_client_and_resource
    client = session.client('s3', config=boto3.session.Config(signature_version='s3v4'), **settings)
  File "/usr/local/lib/python3.10/dist-packages/boto3/session.py", line 299, in client
    return self._session.create_client(
  File "/usr/local/lib/python3.10/dist-packages/botocore/session.py", line 957, in create_client
    credentials = self.get_credentials()
  File "/usr/local/lib/python3.10/dist-packages/botocore/session.py", line 515, in get_credentials
    ).load_credentials()
  File "/usr/local/lib/python3.10/dist-packages/botocore/credentials.py", line 2039, in load_credentials
    creds = provider.load()
  File "/usr/local/lib/python3.10/dist-packages/botocore/credentials.py", line 1902, in load
    return self._retrieve_or_fail()
  File "/usr/local/lib/python3.10/dist-packages/botocore/credentials.py", line 1911, in _retrieve_or_fail
    creds = fetcher()
  File "/usr/local/lib/python3.10/dist-packages/botocore/credentials.py", line 1929, in fetch_creds
    response = self._fetcher.retrieve_full_uri(
  File "/usr/local/lib/python3.10/dist-packages/botocore/utils.py", line 2925, in retrieve_full_uri
    self._validate_allowed_url(full_url)
  File "/usr/local/lib/python3.10/dist-packages/botocore/utils.py", line 2932, in _validate_allowed_url
    raise ValueError(
ValueError: Unsupported host '169.254.170.23'.  Can only retrieve metadata from these hosts: 169.254.170.2, localhost, 127.0.0.1

Describe the solution you'd like

Please upgrade boto/botocore to minimum supported version (1.34.41) https://docs.aws.amazon.com/eks/latest/userguide/pod-id-minimum-sdk.html

Describe alternatives you've considered

Workaround: use (older) IRSA iam role model, or access_key/secret_key.

Thanks!

jombooth commented 4 months ago

/jira create

Workflow run Jira issue TRIAG-753 is created