apache / doris-spark-connector

Spark Connector for Apache Doris
https://doris.apache.org/
Apache License 2.0
83 stars 98 forks source link

Spark Doris Connector Release Note 1.3.0 #159

Open JNSimba opened 1 year ago

JNSimba commented 1 year ago

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

cjj2010 commented 1 year ago

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released image

milton4code commented 1 year ago

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released image

I am the same

lordk911 commented 1 year ago

https://repo.maven.apache.org/maven2/org/apache/doris/spark-doris-connector-3.3_2.12/1.3.0/