linkedin / transport

A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Apache Hive, and Presto.
BSD 2-Clause "Simplified" License
291 stars 72 forks source link

Fix packaging task for Trino with Gradle 7 #147

Closed yiqiangin closed 1 year ago

yiqiangin commented 1 year ago

Code Changes

Tests cd transport ./gradlew clean build cd transportable-udfs-example ./gradlew clean build

test a UDF MP (e.g. standard-udfs-marketing) with a locally-built transport-plugin mint build mint release the content about the artifacts shown in buildspec.json include all jars

"artifacts": {
      "marketing-std-udf": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:hive": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:javadoc": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:sources": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:spark_2.11": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:spark_2.12": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:trino-dist-thin": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7",
      "marketing-std-udf:trino-thin": "ivy:/com.linkedin.standard-udfs-marketing/marketing-std-udf/1.0.7"
    }
aastha25 commented 1 year ago

LGTM! could you link any gradle 7 documentation for reference in the PR description? That would explain how the new configurations enabled jar publishing.

yiqiangin commented 1 year ago

LGTM! could you link any gradle 7 documentation for reference in the PR description? That would explain how the new configurations enabled jar publishing.

This solution was figured out by the engineer from our internal Gradle team. It was recorded in an internal Jira ticket. I am afraid there is no public Gradle 7 documentation about this issue now.

yiqiangin commented 1 year ago

Let's document this process internally for future reusability.

Will do