search

apache / incubator-xtable

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
https://xtable.apache.org/
Apache License 2.0
919 stars 147 forks source link

XTable 1st Release #486

Closed vinishjail97 closed 2 months ago

vinishjail97 commented 4 months ago

Feature Request / Improvement

Copying the email sent to dev@xtable.apache.org https://lists.apache.org/thread/kjwtg55rdcllvyol6sqszd6ozgto7z45 

[DISCUSS] Targeting July Release for Apache XTable (Incubating)
Hi everyone,

I hope this message finds you well.

Based on the guidelines
<https://incubator.apache.org/guides/releasemanagement.html> mentioned for
Apache incubating projects - Podlings can use non-ASF releases as an
opportunity to find ASF policy violations and begin resolving them. I have
put up a release guide PR for the upcoming Apache XTable (Incubating)
release and propose running an interim non-ASF release to iron out issues.
https://github.com/apache/incubator-xtable/pull/434

After resolving issues from the non-ASF release, I propose we target an
ASF-compliant release for Apache XTable (Incubating) in July. Please share
your thoughts and any objections regarding this proposed timeline. Your
feedback is crucial to ensure we meet the necessary requirements and
address any potential issues before the release.

Looking forward to your input.

Best regards,
Vinish

Are you willing to submit PR?

Code of Conduct

vinishjail97 commented 4 months ago

As per suggestions from @jcamachor, @zabetak and @vinothchandar - we will be adding doing an ASF release using the "Work In Progress Disclaimer"

vinishjail97 commented 4 months ago

Pushed PR for DISCLAIMER-WIP -> https://github.com/apache/incubator-xtable/pull/487

vinishjail97 commented 4 months ago

Staged source releases on dis.apache.org https://dist.apache.org/repos/dist/dev/incubator/xtable/

Created PR for adding release profile and mvn deploy plugins -> https://github.com/apache/incubator-xtable/pull/493

Blocker: Not able to upload the artifacts to https://repository.apache.org mvn deploy -Prelease -DskipTests -DdeployArtifacts=true 

[INFO] --- gpg:1.4:sign (sign-artifacts) @ xtable ---
gpg: using "XXXXX" as default secret key for signing
[INFO] 
[INFO] --- install:3.1.1:install (default-install) @ xtable ---
[INFO] Installing /Users/vinishreddy/Opensource/incubator-xtable/pom.xml to /Users/vinishreddy/.m2/repository/org/apache/xtable/xtable/0.1.0-rc1/xtable-0.1.0-rc1.pom
[INFO] Installing /Users/vinishreddy/Opensource/incubator-xtable/target/xtable-0.1.0-rc1.pom.asc to /Users/vinishreddy/.m2/repository/org/apache/xtable/xtable/0.1.0-rc1/xtable-0.1.0-rc1.pom.asc
[INFO] 
[INFO] --- deploy:3.1.1:deploy (default-deploy) @ xtable ---
Uploading to apache.releases.https: https://repository.apache.org/service/local/staging/deploy/maven2/org/apache/xtable/xtable/0.1.0-rc1/xtable-0.1.0-rc1.pom
Uploading to apache.releases.https: https://repository.apache.org/service/local/staging/deploy/maven2/org/apache/xtable/xtable/0.1.0-rc1/xtable-0.1.0-rc1.pom.asc

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:3.1.1:deploy (default-deploy) on project xtable: Failed to deploy artifacts: Could not transfer artifact org.apache.xtable:xtable:pom:0.1.0-rc1 from/to apache.releases.https (https://repository.apache.org/service/local/staging/deploy/maven2): status code: 400, reason phrase: Bad Request (400) -> [Help 1]

I have checked my ~/.m2/settings.xml and ensured my username and password for the nexus repository is populated correctly. I have also verified the artifacts are signed by the gpg key used for staging source releases.

<settings>
    <servers>
        <server>
            <id>apache.releases.https</id>
            <username>$USERNAME</username>
            <password>$PASSWORD</password>
        </server>
        <server>
            <id>apache.snapshots.https</id>
            <username>$USERNAME</username>
            <password>$PASSWORD</password>
        </server>
    </servers>
</settings>

Any help from people who have done releases for incubating projects is appreciated. My guess is someone from Apache needs to approve/create org.apache.incubator-xtable staging profile in https://repository.apache.org/.

vinishjail97 commented 3 months ago

The artifacts have been uploaded to staging repository.

Repository Path: /org/apache/xtable/incubator-xtable-utilities/0.1.0-rc1/incubator-xtable-utilities-0.1.0-rc1-bundled.jar

vinishjail97 commented 3 months ago

Based on comments provided in rc-1, have fixed the artifact names for the bundles that didn't have the incubator prefix.

Created 0.1.0-rc2 which can be accessed through https://repository.apache.org/content/repositories/orgapachextable-1001/

In your pom.xml, you need to make the following changes to access the rc2 bundles.

    <repositories>
        <repository>
            <id>apache-staging</id>
            <name>Apache Staging Repository for XTable</name>
            <url>https://repository.apache.org/content/repositories/orgapachextable-1001/</url>
        </repository>
    </repositories>
    ....
          <dependency>
            <groupId>org.apache.xtable</groupId>
            <artifactId>incubator-xtable-core</artifactId>
            <version>0.1.0-rc2</version>
        </dependency>
vinishjail97 commented 3 months ago

Based on feedback provided by @jcamachor, moving the "incubating" regex to the artifact version instead of the artifact name.

Created 0.1.0-rc3 which can be accessed through https://repository.apache.org/content/repositories/orgapachextable-1002/

In your pom.xml, you need to make the following changes to access the rc3 bundles.

    <repositories>
        <repository>
            <id>apache-staging</id>
            <name>Apache Staging Repository for XTable</name>
            <url>https://repository.apache.org/content/repositories/orgapachextable-1002/</url>
        </repository>
    </repositories>
    ....
          <dependency>
            <groupId>org.apache.xtable</groupId>
            <artifactId>xtable-core</artifactId>
            <version>0.1.0-incubating-rc3</version>
        </dependency>
vinishjail97 commented 3 months ago

Regarding this feedback for rc3, I have updated the release guide to reflect the right source distribution names without the rcX suffix.

https://dist.apache.org/repos/dist/dev/incubator/xtable/0.1.0-incubating-rcX/apache-xtable-0.1.0-incubating-src.tgz
https://dist.apache.org/repos/dist/release/incubator/xtable/0.1.0-incubating/apache-xtable-0.1.0-incubating-src.tgz
In terms of archive names and content the RC suffix should not appear.
Once the vote passes the release artifacts must be moved from the
staging area to the release area unaltered

@zabetak For the above feedback, the maven version needs to exclude the rcX suffix as well ? https://github.com/apache/incubator-xtable/commit/58a327f096a47d27e91ee1efa17982d7c1a75372

Current https://repository.apache.org/content/repositories/orgapachextable-1002/org/apache/xtable/xtable-api/0.1.0-incubating-rc3/xtable-api-0.1.0-incubating-rc3.jar

Recommend. Is this correct ? https://repository.apache.org/content/repositories/orgapachextable-1002/org/apache/xtable/xtable-api/0.1.0-incubating/xtable-api-0.1.0-incubating.jar

https://github.com/apache/incubator-xtable/pull/434#discussion_r1715799164

zabetak commented 3 months ago

@vinishjail97 Regarding the naming convention of the maven artifacts (xtable-api-0.1.0-incubating.jar vs xtable-api-0.1.0-incubating-rc3.jar) I would say that the RC suffix should not be present in the jars. Once the vote passes the artifacts are no longer release candidates so it doesn't make sense to have the suffix there.

For the same reasons I think we should remove the RC suffix from the maven staging repo URL. The staging repo structure should reflect the final state since that is the one released and it cannot be modified after.

Summarizing yes the URL below is the correct one:

https://repository.apache.org/content/repositories/orgapachextable-1002/org/apache/xtable/xtable-api/0.1.0-incubating/xtable-api-0.1.0-incubating.jar
vinishjail97 commented 3 months ago

mvn deploy failed for rc4 branch because of parent pom version upgrade, reverting it back from 33 to 31. https://github.com/apache/incubator-xtable/pull/516

Address issues in the release guide based on rc3 feedback as well -> https://github.com/apache/incubator-xtable/pull/434/commits

vinishjail97 commented 3 months ago

Uploaded rc4 artifacts for 0.1.0-incubating to staging repository.

https://repository.apache.org/content/repositories/orgapachextable-1004/ https://dist.apache.org/repos/dist/dev/incubator/xtable/0.1.0-incubating-rc4/

Release Notes for release-0.1.0-incubating-rc4

This is the first official apache release for Apache XTable, an incubating project under the Apache Software Foundation.Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

Raw Release Notes

Full Changelog: https://github.com/apache/incubator-xtable/compare/v0.1.0-beta1...release-0.1.0-incubating-rc4