search

AbsaOSS / spline-spark-agent

Spline agent for Apache Spark
https://absaoss.github.io/spline/
Apache License 2.0
175 stars 90 forks source link

use splineAgent as a Jar file #761

Open zacayd opened 8 months ago

zacayd commented 8 months ago

Hi i tried to install the Agent Spline without Maven - but Upload of the a Jar FIile i cloned the code from git and build it to a spline-spark-agent_2.12.jar and installed library using Upload to Databricks when i run the code 

import scala.util.parsing.json.JSON
import za.co.absa.spline.harvester.SparkLineageInitializer._
import za.co.absa.spline.agent.AgentConfig
import za.co.absa.spline.harvester.postprocessing.AbstractPostProcessingFilter
import za.co.absa.spline.harvester.postprocessing.PostProcessingFilter
import org.apache.commons.configuration.Configuration
import za.co.absa.spline.harvester.conf.StandardSplineConfigurationStack
import za.co.absa.spline.harvester.HarvestingContext
import za.co.absa.spline.producer.model.ExecutionPlan
import za.co.absa.spline.producer.model.ExecutionEvent
import za.co.absa.spline.producer.model.ReadOperation
import za.co.absa.spline.producer.model.WriteOperation
import za.co.absa.spline.producer.model.DataOperation
import za.co.absa.spline.harvester.ExtraMetadataImplicits._
import za.co.absa.spline.harvester.SparkLineageInitializer._

I get error that it doesnt recognise the packages can you please advise what is missing?

cerveada commented 8 months ago

Hi, share the error, what jar are you uploading?

zacayd commented 8 months ago

HI i have attached the screen shot of the installation of the jar the jar is the artifact of build the project https://github.com/AbsaOSS/spline-spark-agent Screenshot 2023-11-01 130220

the errors are

command-4378807525288898:2: error: not found: value za import za.co.absa.spline.harvester.SparkLineageInitializer. ^ command-4378807525288898:3: error: not found: value za import za.co.absa.spline.agent.AgentConfig ^ command-4378807525288898:4: error: not found: value za import za.co.absa.spline.harvester.postprocessing.AbstractPostProcessingFilter ^ command-4378807525288898:5: error: not found: value za import za.co.absa.spline.harvester.postprocessing.PostProcessingFilter ^ command-4378807525288898:6: error: object configuration is not a member of package org.apache.commons import org.apache.commons.configuration.Configuration ^ command-4378807525288898:7: error: not found: value za import za.co.absa.spline.harvester.conf.StandardSplineConfigurationStack ^ command-4378807525288898:8: error: not found: value za import za.co.absa.spline.harvester.HarvestingContext ^ command-4378807525288898:9: error: not found: value za import za.co.absa.spline.producer.model.ExecutionPlan ^ command-4378807525288898:10: error: not found: value za import za.co.absa.spline.producer.model.ExecutionEvent ^ command-4378807525288898:11: error: not found: value za import za.co.absa.spline.producer.model.ReadOperation ^ command-4378807525288898:12: error: not found: value za import za.co.absa.spline.producer.model.WriteOperation ^ command-4378807525288898:13: error: not found: value za import za.co.absa.spline.producer.model.DataOperation ^ command-4378807525288898:14: error: not found: value za import za.co.absa.spline.harvester.ExtraMetadataImplicits. ^ command-4378807525288898:15: error: not found: value za import za.co.absa.spline.harvester.SparkLineageInitializer._ ^

cerveada commented 8 months ago

If you are modifying the code, you must build and use your own spark-3.3-spline-agent-bundle.jar not the one from maven.

wajda commented 8 months ago

error: not found: value za

It looks like the Jar is either not seen by the Spark, or is empty/does not contain any Spline related classes. Please check your Jar content and the way you attach it to Spark.

zacayd commented 8 months ago

the jar is 5K - it was created when i made a build on the Intelij the file is called spline-spark-agent_2.12.jar i took the content of the POM.xml from Bundle 3.3 I then uploaded the jar as "Install New" on the Databricks Cluster Do you have a working solution of spline for Databricks which is not from Maven Central ?

wajda commented 8 months ago

Please follow the building procedure from this section in the README - https://github.com/AbsaOSS/spline-spark-agent#building Then take the bundle jar from the bundle-3.3/target folder.

zacayd commented 8 months ago

i made this C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5\bin>mvn –version

Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)

Maven home: C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5

Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jre1.8.0_202

Default locale: en_US, platform encoding: Cp1252

OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

But Got

C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5\bin>mvn scala-cross-build:change-version -Pscala-2.11

[INFO] Scanning for projects...

[WARNING] The requested profile "scala-2.11" could not be activated because it does not exist.

Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml

Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml

Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (14 kB at 16 kB/s)

Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (21 kB at 23 kB/s)

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 1.890 s

[INFO] Finished at: 2023-11-06T10:47:47+02:00

[INFO] ------------------------------------------------------------------------

[WARNING] The requested profile "scala-2.11" could not be activated because it does not exist.

[ERROR] No plugin found for prefix 'scala-cross-build' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (C:\Users\zacayd.m2\repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException

From: Alex Vayda @.> Sent: Wednesday, 1 November 2023 18:08 To: AbsaOSS/spline-spark-agent @.> Cc: zacayd @.>; Author @.> Subject: Re: [AbsaOSS/spline-spark-agent] use splineAgent as a Jar file (Issue #761)

error: not found: value za

It looks like the Jar is either not seen by the Spark, or is empty/does not contain any Spline related classes. Please check your Jar content and the way you attach it to Spark.

— Reply to this email directly, view it on GitHub https://github.com/AbsaOSS/spline-spark-agent/issues/761#issuecomment-1789230582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCB5PB4R6PHNXSQ3A2PD53YCJXU7AVCNFSM6AAAAAA6WGALX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGIZTANJYGI . You are receiving this because you authored the thread.Message ID: < @.***>

wajda commented 8 months ago

I've never seen this error. Something might be wrong with your POM files. Try it on a clean project. call the Maven command in the project root directory.

Here's how it should look like:

 ~/Projects/spline-spark-agent │ develop                                                                                                                                                      
❯ mvn scala-cross-build:change-version -Pscala-2.11
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Spline Spark Agent                                                 [pom]
[INFO] agent-commons_2.12                                                 [jar]
[INFO] agent-core_2.12                                                    [jar]
[INFO] examples_2.12                                                      [jar]
[INFO] integration-tests_2.12                                             [jar]
[INFO] spark-2.2-spline-agent-bundle_2.11                                 [jar]
[INFO] spark-2.3-spline-agent-bundle_2.11                                 [jar]
[INFO] spark-2.4-spline-agent-bundle_2.12                                 [jar]
[INFO]
[INFO] -------< za.co.absa.spline.agent.spark:spline-spark-agent_2.12 >--------
[INFO] Building Spline Spark Agent 2.1.0-SNAPSHOT                         [1/8]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spline-spark-agent_2.12 ---
[INFO]
[INFO] ----------< za.co.absa.spline.agent.spark:agent-commons_2.12 >----------
[INFO] Building agent-commons_2.12 2.1.0-SNAPSHOT                         [2/8]
[INFO]   from commons/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-commons_2.12 ---
[INFO]
[INFO] -----------< za.co.absa.spline.agent.spark:agent-core_2.12 >------------
[INFO] Building agent-core_2.12 2.1.0-SNAPSHOT                            [3/8]
[INFO]   from core/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-core_2.12 ---
[INFO]
[INFO] ------------< za.co.absa.spline.agent.spark:examples_2.12 >-------------
[INFO] Building examples_2.12 2.1.0-SNAPSHOT                              [4/8]
[INFO]   from examples/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ examples_2.12 ---
[INFO]
[INFO] --------< za.co.absa.spline.agent.spark:integration-tests_2.12 >--------
[INFO] Building integration-tests_2.12 2.1.0-SNAPSHOT                     [5/8]
[INFO]   from integration-tests/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ integration-tests_2.12 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.2-spline-agent-bundle_2.11 >--
[INFO] Building spark-2.2-spline-agent-bundle_2.11 2.1.0-SNAPSHOT         [6/8]
[INFO]   from bundle-2.2/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.2-spline-agent-bundle_2.11 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.3-spline-agent-bundle_2.11 >--
[INFO] Building spark-2.3-spline-agent-bundle_2.11 2.1.0-SNAPSHOT         [7/8]
[INFO]   from bundle-2.3/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.3-spline-agent-bundle_2.11 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.4-spline-agent-bundle_2.12 >--
[INFO] Building spark-2.4-spline-agent-bundle_2.12 2.1.0-SNAPSHOT         [8/8]
[INFO]   from bundle-2.4/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.4-spline-agent-bundle_2.12 ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spline Spark Agent 2.1.0-SNAPSHOT:
[INFO]
[INFO] Spline Spark Agent ................................. SUCCESS [  0.078 s]
[INFO] agent-commons_2.12 ................................. SUCCESS [  0.009 s]
[INFO] agent-core_2.12 .................................... SUCCESS [  0.009 s]
[INFO] examples_2.12 ...................................... SUCCESS [  0.009 s]
[INFO] integration-tests_2.12 ............................. SUCCESS [  0.009 s]
[INFO] spark-2.2-spline-agent-bundle_2.11 ................. SUCCESS [  0.007 s]
[INFO] spark-2.3-spline-agent-bundle_2.11 ................. SUCCESS [  0.008 s]
[INFO] spark-2.4-spline-agent-bundle_2.12 ................. SUCCESS [  0.009 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.279 s
[INFO] Finished at: 2023-11-06T10:58:25+01:00
[INFO] ------------------------------------------------------------------------
zacayd commented 8 months ago

Hi

I successfully made mvn scala-cross-build:change-version -Pscala-2.11

mvn clean -Pscala-2.11,spark-2.4

also succeeded

but when in run

mvn install -Pscala-2.11,spark-2.4

I got

[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile (scala-compile) on project agent-commons_2.11: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.: CompileFailed -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile (scala-compile) on project agent-commons_2.11: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:333)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo

(DefaultBuildPluginManager.java:133)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:328)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

Caused by: sbt.internal.inc.CompileFailed

at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError

(IncrementalCompilerImpl.scala:335)

at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally

(IncrementalCompilerImpl.scala:420)

at sbt.internal.inc.IncrementalCompilerImpl.compile

(IncrementalCompilerImpl.scala:137)

at sbt_inc.SbtIncrementalCompiler.compile

(SbtIncrementalCompiler.java:179)

at scala_maven.ScalaCompilerSupport.incrementalCompile

(ScalaCompilerSupport.java:356)

at scala_maven.ScalaCompilerSupport.compile

(ScalaCompilerSupport.java:113)

at scala_maven.ScalaCompilerSupport.doExecute

(ScalaCompilerSupport.java:88)

at scala_maven.ScalaMojoSupport.execute (ScalaMojoSupport.java:308)

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo

(DefaultBuildPluginManager.java:126)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:328)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

[ERROR]

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn -rf :agent-commons_2.11

From: Alex Vayda @.> Sent: Monday, 6 November 2023 12:04 To: AbsaOSS/spline-spark-agent @.> Cc: zacayd @.>; Author @.> Subject: Re: [AbsaOSS/spline-spark-agent] use splineAgent as a Jar file (Issue #761)

I've never seen this error. Something might be wrong with your POM files. Try it on a clean project. call the Maven command in the project root directory.

Here's how it should look like:

~/Projects/spline-spark-agent │ develop

❯ mvn scala-cross-build:change-version -Pscala-2.11

[INFO] Scanning for projects...

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Build Order:

[INFO]

[INFO] Spline Spark Agent [pom]

[INFO] agent-commons_2.12 [jar]

[INFO] agent-core_2.12 [jar]

[INFO] examples_2.12 [jar]

[INFO] integration-tests_2.12 [jar]

[INFO] spark-2.2-spline-agent-bundle_2.11 [jar]

[INFO] spark-2.3-spline-agent-bundle_2.11 [jar]

[INFO] spark-2.4-spline-agent-bundle_2.12 [jar]

[INFO]

[INFO] -------< za.co.absa.spline.agent.spark:spline-spark-agent_2.12 >--------

[INFO] Building Spline Spark Agent 2.1.0-SNAPSHOT [1/8]

[INFO] from pom.xml

[INFO] --------------------------------[ pom ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spline-spark-agent_2.12 ---

[INFO]

[INFO] ----------< za.co.absa.spline.agent.spark:agent-commons_2.12 >----------

[INFO] Building agent-commons_2.12 2.1.0-SNAPSHOT [2/8]

[INFO] from commons/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-commons_2.12 ---

[INFO]

[INFO] -----------< za.co.absa.spline.agent.spark:agent-core_2.12 >------------

[INFO] Building agent-core_2.12 2.1.0-SNAPSHOT [3/8]

[INFO] from core/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-core_2.12 ---

[INFO]

[INFO] ------------< za.co.absa.spline.agent.spark:examples_2.12 >-------------

[INFO] Building examples_2.12 2.1.0-SNAPSHOT [4/8]

[INFO] from examples/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ examples_2.12 ---

[INFO]

[INFO] --------< za.co.absa.spline.agent.spark:integration-tests_2.12 >--------

[INFO] Building integration-tests_2.12 2.1.0-SNAPSHOT [5/8]

[INFO] from integration-tests/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ integration-tests_2.12 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.2-spline-agent-bundle_2.11 >--

[INFO] Building spark-2.2-spline-agent-bundle_2.11 2.1.0-SNAPSHOT [6/8]

[INFO] from bundle-2.2/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.2-spline-agent-bundle_2.11 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.3-spline-agent-bundle_2.11 >--

[INFO] Building spark-2.3-spline-agent-bundle_2.11 2.1.0-SNAPSHOT [7/8]

[INFO] from bundle-2.3/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.3-spline-agent-bundle_2.11 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.4-spline-agent-bundle_2.12 >--

[INFO] Building spark-2.4-spline-agent-bundle_2.12 2.1.0-SNAPSHOT [8/8]

[INFO] from bundle-2.4/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.4-spline-agent-bundle_2.12 ---

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary for Spline Spark Agent 2.1.0-SNAPSHOT:

[INFO]

[INFO] Spline Spark Agent ................................. SUCCESS [ 0.078 s]

[INFO] agent-commons_2.12 ................................. SUCCESS [ 0.009 s]

[INFO] agent-core_2.12 .................................... SUCCESS [ 0.009 s]

[INFO] examples_2.12 ...................................... SUCCESS [ 0.009 s]

[INFO] integration-tests_2.12 ............................. SUCCESS [ 0.009 s]

[INFO] spark-2.2-spline-agent-bundle_2.11 ................. SUCCESS [ 0.007 s]

[INFO] spark-2.3-spline-agent-bundle_2.11 ................. SUCCESS [ 0.008 s]

[INFO] spark-2.4-spline-agent-bundle_2.12 ................. SUCCESS [ 0.009 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 0.279 s

[INFO] Finished at: 2023-11-06T10:58:25+01:00

[INFO] ------------------------------------------------------------------------

— Reply to this email directly, view it on GitHub https://github.com/AbsaOSS/spline-spark-agent/issues/761#issuecomment-1794472867, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCB5PGEQX7CIOJH2C274TTYDCY2NAVCNFSM6AAAAAA6WGALX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJUGQ3TEOBWG4 . You are receiving this because you authored the thread.Message ID: < @.***>

wajda commented 7 months ago

There seems to be a compilation error. Are you compiling the original source code, or you changed something in it?

zacayd commented 7 months ago

the origanl code

wajda commented 7 months ago

Sorry I cannot reproduce it. It builds fine on all our environments including CI and personal PCs. Please try to build it on another env. It must work.