Closed NhatDuy11 closed 10 months ago
currently i am using : iceberg-kafka-connect-runtime-hive-0.6.2
You have set iceberg.s3a.endpoint
, did you mean to set iceberg.catalog.s3.endpoint
?
@bryanck Hi, I also tried this case iceberg.s3.endpoint and got this error: Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])): Profile file contained no credentials for profile 'default': ProfileFile(profilesAndSectionsMap=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
@bryanck thank you ,I have sink the data into minio,because I am missing the config credentials of the aws service.
Were you able to get this working?
I'm going to close this, feel free to reopen or open a new ticket if you have new information.
Here is my iceberg sink connector configuration: { { "name": "IcebergSinkConnectorConnector_4", "config": { "iceberg.catalog.s3.secret-access-key": "###########", "iceberg.catalog.io-impl": "org.apache.iceberg.aws.s3.S3FileIO", "iceberg.catalog.credential": "software.amazon.awssdk.auth.credentials.SimpleAwsCredentialsProvider", "bootstrap-servers": "localhost:9092", "iceberg.catalog.client.region": "us-east-1", "iceberg.catalog.uri": "thrift://hive:9083", "iceberg.s3a.endpoint": "http://192.168.1.206", "value.converter.schema.registry.url": "http://localhost:8081", "iceberg.catalog.warehouse": "s3a://data/warehouse", "key.converter.schema.registry.url": "http://localhost:8081", "iceberg.catalog.s3.access-key-id": "########", "name": "IcebergSinkConnectorConnector_4", "connector.class": "io.tabular.iceberg.connect.IcebergSinkConnector", "tasks.max": "1", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "value.converter": "io.confluent.connect.avro.AvroConverter", "topics": "TIME_ICEBERG", "iceberg.tables": "cnd.iceberg_time"
} } Can someone help me with this error: Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: S3, Status Code: 403, Request ID: EGWB54QFPR4KKS3N, Extended Request ID: Q0PLcS077oYiTKn1he01+Rjm57iKdtd98FQ5Ef4no1pMiBWLZM+nTeeTKGR4aFsOXVs6VYe4Q38=) currently i am using : iceberg-kafka-connect-runtime-hive-0.6.2 Thank you everyone for reading !