CODAIT / stocator

Stocator is high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.
Apache License 2.0
112 stars 72 forks source link

build failing with java classes not found #327

Closed nkurn closed 1 year ago

nkurn commented 1 year ago

Hi, I have echo $JAVA_HOME /Library/Java/JavaVirtualMachines/amazon-corretto-15.jdk/Contents/Home

I cloned the repo

git clone https://github.com/CODAIT/stocator.git Cloning into 'stocator'... remote: Enumerating objects: 8660, done. remote: Counting objects: 100% (630/630), done. remote: Compressing objects: 100% (253/253), done. remote: Total 8660 (delta 247), reused 558 (delta 201), pack-reused 8030 Receiving objects: 100% (8660/8660), 1.72 MiB | 7.38 MiB/s, done. Resolving deltas: 100% (3519/3519), done.

cd stocator

I am trying to build the jar with dependencies using

mvn clean package -Pall-in-one

I am getting error java classes not found, how do I fix the error

mvn clean package -Pall-in-one [INFO] Scanning for projects... [INFO] [INFO] ---------------------< com.ibm.stocator:stocator >---------------------- [INFO] Building com.ibm.stocator:stocator 1.1.6-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [WARNING] Parameter 'excludes' is unknown for plugin 'maven-assembly-plugin:2.3:single (make-assembly)' [INFO] [INFO] --- clean:3.2.0:clean (default-clean) @ stocator --- [INFO] [INFO] --- checkstyle:3.1.0:check (checkstyle) @ stocator --- [INFO] Starting audit... Audit done. [INFO] [INFO] --- resources:3.3.0:resources (default-resources) @ stocator --- [WARNING] File encoding has not been set, using platform encoding UTF-8. Build is platform dependent! [WARNING] See https://maven.apache.org/general.html#encoding-warning [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] The encoding used to copy filtered properties files have not been set. This means that the same encoding will be used to copy filtered properties files as when copying other filtered resources. This might not be what you want! Run your build with --debug to see which files might be affected. Read more at https://maven.apache.org/plugins/maven-resources-plugin/examples/filtering-properties-files.html [INFO] [INFO] --- compiler:3.8.1:compile (default-compile) @ stocator --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 53 source files to /Users/KURNENA/jira/stocator/target/classes [INFO] /Users/KURNENA/jira/stocator/src/main/java/com/ibm/stocator/fs/cos/COSAPIClient.java: Some input files use or override a deprecated API. [INFO] /Users/KURNENA/jira/stocator/src/main/java/com/ibm/stocator/fs/cos/COSAPIClient.java: Recompile with -Xlint:deprecation for details. [INFO] /Users/KURNENA/jira/stocator/src/main/java/com/ibm/stocator/fs/cos/COSAPIClient.java: /Users/KURNENA/jira/stocator/src/main/java/com/ibm/stocator/fs/cos/COSAPIClient.java uses unchecked or unsafe operations. [INFO] /Users/KURNENA/jira/stocator/src/main/java/com/ibm/stocator/fs/cos/COSAPIClient.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] >>> findbugs:3.0.5:check (default) > :findbugs @ stocator >>> [INFO] [INFO] --- findbugs:3.0.5:findbugs (findbugs) @ stocator --- WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/Users/KURNENA/.m2/repository/org/codehaus/groovy/groovy/2.4.12/groovy-2.4.12.jar) to method java.lang.Object.finalize() WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [INFO] Fork Value is true [java] WARNING: An illegal reflective access operation has occurred [java] WARNING: Illegal reflective access by org.dom4j.io.SAXContentHandler (file:/Users/KURNENA/.m2/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar) to method com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser$LocatorProxy.getEncoding() [java] WARNING: Please consider reporting this to the maintainers of org.dom4j.io.SAXContentHandler [java] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations [java] WARNING: All illegal access operations will be denied in a future release [java] The following classes needed for analysis were missing: [java] java.lang.Object [java] java.util.concurrent.Callable [java] java.io.Closeable [java] java.lang.Enum [java] java.lang.Runnable [java] java.io.OutputStream [java] java.util.concurrent.ThreadFactory [java] java.io.InputStream [java] java.util.concurrent.RejectedExecutionHandler [java] java.io.ByteArrayOutputStream [java] java.io.FilterInputStream [java] java.io.IOException [java] java.lang.RuntimeException [java] java.lang.Comparable [java] java.io.Serializable [java] java.io.ObjectInputValidation [java] java.util.concurrent.ExecutorService [java] java.lang.Long [java] java.lang.StringBuilder [java] java.lang.Math [java] java.io.EOFException [java] java.util.Map [java] java.util.concurrent.atomic.AtomicBoolean [java] java.lang.String [java] java.io.FileOutputStream [java] java.io.BufferedOutputStream [java] java.io.File [java] java.lang.Boolean [java] java.lang.InterruptedException [java] java.io.InterruptedIOException [java] java.lang.IllegalStateException [java] java.util.Locale [java] java.net.URI [java] java.util.UUID [java] java.util.concurrent.Semaphore [java] java.lang.Thread [java] java.nio.file.AccessDeniedException [java] java.io.FileNotFoundException [java] java.util.concurrent.ExecutionException [java] java.lang.Throwable [java] java.util.Date [java] java.net.URLDecoder [java] java.io.UnsupportedEncodingException [java] java.lang.IndexOutOfBoundsException [java] java.lang.IllegalArgumentException [java] java.util.Properties [java] java.net.URISyntaxException [java] java.nio.charset.Charset [java] java.text.SimpleDateFormat [java] java.lang.System [java] java.util.regex.Pattern [java] java.util.regex.Matcher [java] java.lang.Integer [java] java.util.ArrayList [java] java.util.Arrays [java] java.util.Iterator [java] java.lang.Class [java] java.util.concurrent.CountDownLatch [java] java.util.HashMap [java] java.lang.ClassLoader [java] java.lang.reflect.Constructor [java] java.lang.Character [java] java.util.Collection [java] java.net.URL [java] java.net.URLEncoder [java] java.lang.Exception [java] java.util.regex.PatternSyntaxException [java] java.util.concurrent.TimeUnit [java] java.util.concurrent.ConcurrentHashMap [java] java.util.concurrent.LinkedBlockingQueue [java] java.util.concurrent.ThreadPoolExecutor [java] java.util.List [java] java.io.PipedInputStream [java] java.lang.SecurityManager [java] java.util.concurrent.atomic.AtomicInteger [java] java.util.Set [java] java.util.Map$Entry [java] java.lang.ThreadGroup [java] java.io.ByteArrayInputStream [java] java.util.concurrent.Executors [java] java.io.PipedOutputStream [java] java.util.concurrent.Future [java] java.io.FileInputStream [java] java.util.LinkedList [java] java.util.Queue [java] java.util.StringTokenizer [java] java.lang.NoSuchFieldError [java] java.lang.UnsatisfiedLinkError [java] java.lang.IllegalAccessError [java] java.lang.IncompatibleClassChangeError [java] java.net.SocketTimeoutException [java] javax.net.ssl.SSLException [java] java.lang.AbstractMethodError [java] java.lang.ArrayIndexOutOfBoundsException [java] java.lang.Error [java] java.lang.NullPointerException [java] java.lang.ClassFormatError [java] java.lang.VerifyError [java] java.lang.ExceptionInInitializerError [java] java.lang.NegativeArraySizeException [java] java.nio.charset.StandardCharsets [java] java.text.ParseException [java] java.util.concurrent.AbstractExecutorService [java] java.lang.InstantiationException [java] java.lang.IllegalAccessException [java] java.lang.NoSuchMethodException [java] java.lang.SecurityException [java] java.net.HttpURLConnection [java] java.lang.ReflectiveOperationException [java] Missing classes: 90 [INFO] Done FindBugs Analysis.... [INFO] [INFO] <<< findbugs:3.0.5:check (default) < :findbugs @ stocator <<< [INFO] [INFO] [INFO] --- findbugs:3.0.5:check (default) @ stocator --- [INFO] BugInstance size is 0 [INFO] Error size is 0 [INFO] No errors/warnings found [INFO] [INFO] --- resources:3.3.0:testResources (default-testResources) @ stocator --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- compiler:3.8.1:testCompile (default-testCompile) @ stocator --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 33 source files to /Users/KURNENA/jira/stocator/target/test-classes [INFO] /Users/KURNENA/jira/stocator/src/test/java/com/ibm/stocator/fs/common/FileSystemTestUtils.java: Some input files use or override a deprecated API. [INFO] /Users/KURNENA/jira/stocator/src/test/java/com/ibm/stocator/fs/common/FileSystemTestUtils.java: Recompile with -Xlint:deprecation for details. [INFO] [INFO] --- surefire:2.22.2:test (default-test) @ stocator --- [WARNING] Parameter 'localRepository' is deprecated core expression; Avoid use of ArtifactRepository type. If you need access to local repository, switch to '${repositorySystemSession}' expression and get LRM from it instead. [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemBasicOps log4j:WARN No appenders could be found for logger (com.ibm.stocator.fs.common.FileSystemTestUtils). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 7, Time elapsed: 0.259 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemBasicOps [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemLsOperations [WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.001 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemLsOperations [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemDelete [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemDelete [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemRead [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0.001 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemRead [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.ObjectStoreFileSystemTest [WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 0.003 s - in com.ibm.stocator.fs.swift2d.systemtests.ObjectStoreFileSystemTest [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemExtendedContract [WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 7, Time elapsed: 0.001 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemExtendedContract [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.StreamingSwiftTest [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.011 s - in com.ibm.stocator.fs.swift2d.systemtests.StreamingSwiftTest [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.CollisionTest [WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.001 s - in com.ibm.stocator.fs.swift2d.systemtests.CollisionTest [INFO] Running com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemBlocksize [WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 0.001 s - in com.ibm.stocator.fs.swift2d.systemtests.TestSwiftFileSystemBlocksize [INFO] Running com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest [ERROR] Tests run: 8, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 1.931 s <<< FAILURE! - in com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest [ERROR] isJobSuccessfulTest(com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest) Time elapsed: 0.076 s <<< ERROR! java.lang.NoClassDefFoundError: com/sun/activation/registries/LogSupport at com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest.isJobSuccessfulTest(SwiftAPIClientTest.java:193) Caused by: java.lang.ClassNotFoundException: com.sun.activation.registries.LogSupport at com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest.isJobSuccessfulTest(SwiftAPIClientTest.java:193)

[ERROR] setCorrectSizeTest(com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest) Time elapsed: 0.002 s <<< ERROR! java.lang.NoClassDefFoundError: com/sun/activation/registries/LogSupport at com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest.setCorrectSizeTest(SwiftAPIClientTest.java:224)

[ERROR] getFileStatusTest(com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest) Time elapsed: 0.001 s <<< ERROR! java.lang.NoClassDefFoundError: com/sun/activation/registries/LogSupport at com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest.getFileStatusTest(SwiftAPIClientTest.java:239)

[ERROR] isSparkOriginTest(com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest) Time elapsed: 0.002 s <<< ERROR! java.lang.NoClassDefFoundError: com/sun/activation/registries/LogSupport at com.ibm.stocator.fs.swift2d.unittests.SwiftAPIClientTest.isSparkOriginTest(SwiftAPIClientTest.java:264)

[INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberSpecialChars [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberSpecialChars [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestEmptyObjectFlatListing [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.001 s - in com.ibm.stocator.fs.cos.systemtests.TestEmptyObjectFlatListing [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestListing [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestListing [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestListingDeepHivePartition [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestListingDeepHivePartition [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberBracket [WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 0.001 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberBracket [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSFaultToleranceCleanupMode [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSFaultToleranceCleanupMode [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSObjectStoreFS [WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSObjectStoreFS [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestListingDeepHivePartitionLevels [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestListingDeepHivePartitionLevels [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSGlobber [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSGlobber [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestAtomicWrite [WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed: 0.001 s - in com.ibm.stocator.fs.cos.systemtests.TestAtomicWrite [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberBracketStocator [WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 7, Time elapsed: 0.001 s - in com.ibm.stocator.fs.cos.systemtests.TestCOSGlobberBracketStocator [INFO] Running com.ibm.stocator.fs.cos.systemtests.TestEmptyObject [WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0 s - in com.ibm.stocator.fs.cos.systemtests.TestEmptyObject [INFO] Running com.ibm.stocator.fs.cos.tests.COSAPIClientTest [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.985 s <<< FAILURE! - in com.ibm.stocator.fs.cos.tests.COSAPIClientTest [ERROR] innerQualifyTest(com.ibm.stocator.fs.cos.tests.COSAPIClientTest) Time elapsed: 0.84 s <<< ERROR! java.lang.RuntimeException: Internal error: Failed to find the "modifiers" field in method setInternalState. at com.ibm.stocator.fs.cos.tests.COSAPIClientTest.before(COSAPIClientTest.java:52) Caused by: java.lang.NoSuchFieldException: modifiers at com.ibm.stocator.fs.cos.tests.COSAPIClientTest.before(COSAPIClientTest.java:52)

[ERROR] pathToKeyTest(com.ibm.stocator.fs.cos.tests.COSAPIClientTest) Time elapsed: 0 s <<< ERROR! java.lang.RuntimeException: Internal error: Failed to find the "modifiers" field in method setInternalState. at com.ibm.stocator.fs.cos.tests.COSAPIClientTest.before(COSAPIClientTest.java:52) Caused by: java.lang.NoSuchFieldException: modifiers at com.ibm.stocator.fs.cos.tests.COSAPIClientTest.before(COSAPIClientTest.java:52)

[INFO] Running com.ibm.stocator.fs.common.unittests.StocatorPathQualifyTest log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.053 s <<< FAILURE! - in com.ibm.stocator.fs.common.unittests.StocatorPathQualifyTest [ERROR] isTempPathTest(com.ibm.stocator.fs.common.unittests.StocatorPathQualifyTest) Time elapsed: 0.983 s <<< ERROR! java.lang.RuntimeException: Internal error: Failed to find the "modifiers" field in method setInternalState. at com.ibm.stocator.fs.common.unittests.StocatorPathQualifyTest.before(StocatorPathQualifyTest.java:71) Caused by: java.lang.NoSuchFieldException: modifiers at com.ibm.stocator.fs.common.unittests.StocatorPathQualifyTest.before(StocatorPathQualifyTest.java:71)

[INFO] Running com.ibm.stocator.fs.common.unittests.StocatorPathTest log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.905 s - in com.ibm.stocator.fs.common.unittests.StocatorPathTest [INFO] Running com.ibm.stocator.fs.common.unittests.CommonUtilsTest [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.ibm.stocator.fs.common.unittests.CommonUtilsTest [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] StocatorPathQualifyTest.before:71 » Runtime Internal error: Failed to find the... [ERROR] COSAPIClientTest.before:52 » Runtime Internal error: Failed to find the "modif... [ERROR] COSAPIClientTest.before:52 » Runtime Internal error: Failed to find the "modif... [ERROR] SwiftAPIClientTest.getFileStatusTest:239 » NoClassDefFound com/sun/activation/... [ERROR] SwiftAPIClientTest.isJobSuccessfulTest:193 » NoClassDefFound com/sun/activatio... [ERROR] SwiftAPIClientTest.isSparkOriginTest:264 » NoClassDefFound com/sun/activation/... [ERROR] SwiftAPIClientTest.setCorrectSizeTest:224 » NoClassDefFound com/sun/activation... [INFO] [ERROR] Tests run: 100, Failures: 0, Errors: 7, Skipped: 79 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 21.340 s [INFO] Finished at: 2023-04-22T19:53:18-04:00

nkurn commented 1 year ago

it can't find jave classes, not sure how to fix this, can someone please suggest a solution?

thanks.

nkurn commented 1 year ago

I am building this on MAC

mrmadira commented 1 year ago

please try java1.8

nkurn commented 1 year ago

is java 1.8 available for community? can you please provide the link for MAC if you have it?

thanks!

mrmadira commented 1 year ago

Here's how i do from my Eclipse on Mac

image image
nkurn commented 1 year ago

thank you so much, that worked!!