Open sharon-clue opened 9 months ago
same issue here, i think it's not only a matter of removing the required fields as the COPY command is explicitly using credentials (i guess it would be needed to add the service account name there)
the service account is attached to the pod, so no need to do anything besides allowing the whole thing work without having the secret/access key. If i have time i'll change it and create a pull request
Same issue here. Do you have an estimated timeframe for this? I would love to help, but I know nothing about Java 😩
I'm facing the same problem using k8s service accounts only. Looking here I think this is not possible, because the function expects a access/secret key environment variables.
I'm getting this error
Stack Trace: java.util.NoSuchElementException: Key AWS_ACCESS_KEY_ID is missing in the map.
at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapWithDefault.kt:24)
at kotlin.collections.MapsKt__MapsKt.getValue(Maps.kt:360)
at io.airbyte.cdk.integrations.destination.s3.credential.S3AssumeRoleCredentialConfig$Companion.getCredentialProvider(S3AssumeRoleCredentialConfig.kt:65)
@stephane-airbyte can you confirm please?
For the moment, we don't have answers to these questions, as we are focused on enabling this feature for users of Airbyte Cloud. We will later explore if/how we want to enable this for self-hosted users. https://github.com/airbytehq/airbyte/pull/39371
Thanks for the clarification @evantahler
Hello @evantahler, I can see that you are tackling this for S3 destination, however the same applies to Redshift destination with COPY strategy (S3 Staging), it makes sense enabling passing ARN Role for both, WDYT? Especially that we only have that option now that standard inserts have been removed
@evantahler I see that your PR is closed in favor of this PR https://github.com/airbytehq/airbyte/pull/39468 but the new PR is still open. Is the connector with Role ARN authentication still not available for Airbyte OSS? Is it available for Airbyte Enterprise?
Connector Name
destination-redshift
Connector Version
0.6.11
What step the error happened?
Configuring a new connector
Relevant information
I'm trying to setup the connector while using the
S3 Staging
option. the aws key-id and access are mandatory and it's not required since my my implementation is using a service account that has the needed permissions to the s3 bucket.please change it to allow using the
s3 staging
option without the aws key-id and access keyRelevant log output
No response
Contribute