search

databricks / simr

Spark In MapReduce (SIMR) - launching Spark applications on existing Hadoop MapReduce infrastructure
http://databricks.github.io/simr/
45 stars 19 forks source link

java.lang.UnsatisfiedLinkError on Hadoop 2.0.0-cdh4.2.1 #24

Open uokuyucu opened 10 years ago

uokuyucu commented 10 years ago

I have CDH4 Hadoop installation. Followed instructions on the SIMR page, tried running simr --shell for the first time and got the following stack trace:

Uploading simr-cdh4.jar (439K) to the Hadoop cluster, this may take a while
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/geo/data/uokuyucu/simr/spark-assembly-hadoop-cdh4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Did not specifiy outdir, trying: simr-2014-10-11_23_06_10
Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
    at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
    at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
    at org.apache.hadoop.security.UserGroupInformation.initUGI(UserGroupInformation.java:252)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:223)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
    at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:277)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:668)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:573)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2464)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2456)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2323)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:163)
    at org.apache.spark.simr.SimrJob.updateConfig(SimrJob.java:181)
    at org.apache.spark.simr.SimrJob.run(SimrJob.java:293)
    at org.apache.spark.simr.SimrJob.main(SimrJob.java:338)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
    ... 21 more
Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
    at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method)
    at org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
    at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
    ... 26 more
Bugg-s commented 9 years ago

how to resolve the error??? anybody can help me.... i hv the same problem during spark building in ubuntu 14.04! There is not possibility to compatible problem, bcoz doign prjct in ubuntu 14.04 along the same path of scala and java too.