snowflakedb / snowflake-ingest-java

Java SDK for the Snowflake Ingest Service -
http://www.snowflake.net
Apache License 2.0
71 stars 57 forks source link

SNOW-1727532 Set number of values for repeated fields #861

Closed sfc-gh-alhuang closed 3 weeks ago

sfc-gh-alhuang commented 1 month ago

This PR includes following change:

  1. Set numberOfValues for repeated fields (e.g. map, list) and used it in EP file registration.
  2. Increment null count for all sub-columns of a null column.
  3. Use hash for approximate NDV for string and bytes.
  4. Fix NPE when inserting null to a structured column, throwing SF custom exception instead.
  5. Added extendedMetadataSize and metadataSize fields for EP registration.
  6. Address comments of #851