snowflakedb / snowflake-ingest-java

Java SDK for the Snowflake Ingest Service -
http://www.snowflake.net
Apache License 2.0
65 stars 51 forks source link

DO NOT MERGE: Make binary string encoding configurable, add support for base64 #770

Open sfc-gh-kgaputis opened 1 month ago

sfc-gh-kgaputis commented 1 month ago

Addresses issue where streaming client receives base64-encoded strings for binary.

Adds new parameter BINARY_STRING_ENCODING which supports values hex and base64. The default value is hex.

sfc-gh-tzhang commented 1 month ago

@sfc-gh-lsembera what do you think about supporting base64 as well?

Also, could we do it automatically instead of adding a parameter?

sfc-gh-tzhang commented 1 month ago

Also, could we do it automatically instead of adding a parameter?

We cannot do it automatically. Without additional information, we cannot tell if a string is base64- or hex-encoded.

Thanks for confirming, then I think this should be a channel level configuration instead of the whole client, take a look at what we did for setDefaultTimezone

sfc-gh-lsembera commented 1 month ago

Thanks for confirming, then I think this should be a channel level configuration instead of the whole client, take a look at what we did for setDefaultTimezone

The problem with having the configuration per-channel is that it is not as easily configurable from KC.