search

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
http://h2o.ai
Apache License 2.0
6.94k stars 2k forks source link

H2O failing start-up on Apache Hadoop 3.0.0-alpha3 #11534

Open exalate-issue-sync[bot] opened 1 year ago

exalate-issue-sync[bot] commented 1 year ago

Running : hadoop jar h2o.jar -ice_root /data -flow_dir /flows

On Apache Hadoop 3.0.0-alpha3 (it seems the only version affected) Following error occurs: 2017-07-02 09:32:35,044 INFO java.NativeLibrary: Loaded library from lib/linux_64/libxgboost4j_gpu.so (/tmp/libxgboost4j_gpu1041391911743955741.so) 2017-07-02 09:32:36,551 INFO util.log: Logging initialized @6202ms Exception in thread "main" java.lang.NoSuchMethodError: org.eclipse.jetty.server.AbstractConnector: method ()V not found at org.eclipse.jetty.server.bio.SocketConnector.(SocketConnector.java:70) at water.AbstractHTTPD.startHttp(AbstractHTTPD.java:198) at water.AbstractHTTPD.start(AbstractHTTPD.java:90) at water.init.NetworkInit.initializeNetworkSockets(NetworkInit.java:95) at water.H2O.startLocalNode(H2O.java:1470) at water.H2O.main(H2O.java:1959) at water.H2OStarter.start(H2OStarter.java:21) at water.H2OStarter.start(H2OStarter.java:42) at water.H2OApp.main(H2OApp.java:9) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:239) at org.apache.hadoop.util.RunJar.main(RunJar.java:153)

exalate-issue-sync[bot] commented 1 year ago

Michal Malohlava commented: Hi [~accountid:557058:8ce968ef-11d8-4f54-87b4-b400d385ae5c],

may you please try to reproduce this one?

Suspicious points:

exalate-issue-sync[bot] commented 1 year ago

Patrick Rice commented: This is because they launched using h2o instead of h2odriver

I do, however, still get an exception when launching h2odriver I'm assuming this is because we currently do not have a build for hadoop 3.0.0 and the abi is not identical

This is the exception I get when using an h2odriver

Exception in thread "main" java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegator at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3172) at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3217) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3256) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3307) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3275) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:225) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:460) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:533) at water.hadoop.h2odriver.run2(h2odriver.java:1539) at water.hadoop.h2odriver.run(h2odriver.java:1695) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at water.hadoop.h2odriver.main(h2odriver.java:1733) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:239) at org.apache.hadoop.util.RunJar.main(RunJar.java:153)

hasithjp commented 1 year ago

JIRA Issue Migration Info

Jira Issue: PUBDEV-4658 Assignee: Patrick Rice Reporter: Fabrizio Torelli State: Open Fix Version: N/A Attachments: N/A Development PRs: N/A