SNOW-1497358 Support multiple storage for Iceberg mode

snowflakedb / snowflake-ingest-java

Java SDK for the Snowflake Ingest Service -

http://www.snowflake.net

Apache License 2.0

69 stars 54 forks source link

Closed sfc-gh-alhuang closed 2 months ago

sfc-gh-alhuang commented 3 months ago

This PR includes several refactoring for FlushService and StreamingIngestInternalStage to support Iceberg/Non-Iceberg mode file upload

Create ConfigureCallHandler for different configure call includes per client (Non-Iceberg) and per channel (Iceberg mode).
Create StorageManager to manage storage and blob path generation.

Tests for Iceberg mode will be added once the refactor review is done.