apache / orc

Apache ORC - the smallest, fastest columnar storage for Hadoop workloads
https://orc.apache.org/
Apache License 2.0
689 stars 483 forks source link

ORC-1545: [JAVA] Use `orc-format` 1.0.0-SNAPSHOT #1683

Closed dongjoon-hyun closed 11 months ago

dongjoon-hyun commented 11 months ago

What changes were proposed in this pull request?

This PR aims to use orc-format 1.0.0-SNAPSHOT in Java as an interim stage.

  1. We need to change C++ separately later.
  2. After (1), we can delete proto/orc_proto.proto.
  3. After releasing orc-format 1.0.0, we need to use it instead of SNAPSHOT.

Why are the changes needed?

To verify orc-format repository and snapshot.

How was this patch tested?

Pass the CIs.

dongjoon-hyun commented 11 months ago

cc @williamhyun , @wgtmac , @guiyanakuang, @mystic-lama

dongjoon-hyun commented 11 months ago

Merged to main for Apache ORC 2.0.0

paliwalashish commented 11 months ago

Thanks @dongjoon-hyun

paliwalashish commented 11 months ago

@dongjoon-hyun Do we need the shading section on ORC side? We are creating same files on orc-format build too. I am not too sure, so thought of asking

https://github.com/apache/orc/blob/54e28b0b9a83f79932c00427fb0203af257d7d71/java/pom.xml#L489-L512

dongjoon-hyun commented 11 months ago

To @mystic-lama , yes. I was confused like you. Please see here.

Since orc-format repo only shades orc-format*.jar, orc repo should shade orc-core, orc-mapreduce, and orc-shims with the same pattern consistently. Otherwise, the downstream cannot find APIs.

- orc-core/1.9.2/shaded-protobuf/orc-core-1.9.2-shaded-protobuf.jar
- orc-mapreduce/1.9.2/shaded-protobuf/orc-mapreduce-1.9.2-shaded-protobuf.jar
- orc-shims/1.9.2//orc-shims-1.9.2.jar
+ orc-core/2.0.0-SNAPSHOT/shaded-protobuf/orc-core-2.0.0-SNAPSHOT-shaded-protobuf.jar
+ orc-format/1.0.0-SNAPSHOT/shaded-protobuf/orc-format-1.0.0-SNAPSHOT-shaded-protobuf.jar
+ orc-mapreduce/2.0.0-SNAPSHOT/shaded-protobuf/orc-mapreduce-2.0.0-SNAPSHOT-shaded-protobuf.jar
+ orc-shims/2.0.0-SNAPSHOT//orc-shims-2.0.0-SNAPSHOT.jar
paliwalashish commented 11 months ago

@dongjoon-hyun Thanks ! That explains it well. I was assuming to be the case, the concrete example helps.