search

DataSQRL / sqrl

Flexible development framework for building streaming data applications in SQL with Kafka, Flink, Postgres, GraphQL, and more.
https://www.datasqrl.com/
97 stars 14 forks source link

Classpath Generation Error with Fat JAR Naming Convention in a Function package #927

Closed nandorsoma closed 1 month ago

nandorsoma commented 1 month ago

There is an issue with the classpath generation for the Function JAR files. When the JAR is named sqrl-math-0.5.0-SNAPSHOT-jar-with-dependencies.jar, the classpath is generated incorrectly, leading to runtime errors when trying to load the library. However, when the JAR is named sqrl-math-0.5.0-SNAPSHOT.jar, the classpath is generated correctly, and the library functions as expected.

org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.datasqrl.math.util.cbrt
ClassLoader info: URL ClassLoader:
    file: '/build/build/deploy/flink/lib/sqrl-math-0.5.0-snapshot.jar,file:/build/build/deploy/flink/lib/sqrl-math-0.5.0-snapshot-jar-with-dependencies.jar' (missing)
Class not resolvable through given classloader.
henneberger commented 1 month ago

You must add all dependencies as jar files or package it as an uber jar. It's recommended that all UDFs be self contained and packaged as uber jars, with dependencies shaded if they could conflict.