According to the documentation from iceberg sink, creating iceberg sink only requires one of S3.endpoint or S3.region. However, even when S3.endpoint is specified, S3.region is still required.
Error message/log
on #7ba6650, create iceberg sink shows:
ERROR: Failed to run the query
Caused by these errors (recent errors listed first):
1: Sink error
2: Iceberg error
3: Unexpected => IO operation failed, source
4: ConfigInvalid (permanent) at Builder::build, context: { service: s3 } => region is missing. Please find it by S3::detect_region() or set them in env.
docker-compose up
docker-compose exec -it spark-iceberg spark-sql
CREATE TABLE IF NOT EXISTS demo.dev.sbtest1_sink (
id BIGINT,
k INT,
c STRING,
pad STRING
) TBLPROPERTIES (
'format-version' = '2'
);
Create Iceberg Sink
start risingwave
./risedev dev
connect to risingwave with psql
run sql
SET STREAMING_PARALLELISM=1;
CREATE SINK IF NOT EXISTS sbtest1
FROM sbtest1_sink
WITH (
connector = 'iceberg',
type = 'upsert',
primary_key = 'id',
warehouse.path ='s3a://warehouse',
s3.endpoint = 'http://localhost:9000',
s3.access.key = 'admin',
s3.secret.key = 'password',
catalog.name = 'demo',
database.name='dev',
table.name='sbtest1_sink',
);
Get error message:
ERROR: Failed to run the query
Caused by these errors (recent errors listed first):
1: Sink error
2: Iceberg error
3: Unexpected => IO operation failed, source
4: ConfigInvalid (permanent) at Builder::build, context: { service: s3 } => region is missing. Please find it by S3::detect_region() or set them in env.
### Expected behavior
Sink created successfully.
### How did you deploy RisingWave?
using risedev.
```bash
./risedev dev
Describe the bug
According to the documentation from iceberg sink, creating iceberg sink only requires one of
S3.endpoint
orS3.region
. However, even whenS3.endpoint
is specified,S3.region
is still required.Error message/log
To Reproduce
Deploy Iceberg on S3 with MinIO and Spark
create docker-compose file:
start the compose, and then enter the SQL shell.
Create Iceberg Sink
psql
Get error message:
Caused by these errors (recent errors listed first): 1: Sink error 2: Iceberg error 3: Unexpected => IO operation failed, source 4: ConfigInvalid (permanent) at Builder::build, context: { service: s3 } => region is missing. Please find it by S3::detect_region() or set them in env.
The version of RisingWave
dev=> select version(); version
PostgreSQL 13.14.0-RisingWave-2.2.0-alpha (unknown) (1 row)
Additional context
No response