search

paulhoule / telepath

System for mining Wikipedia Usage data to read our collective mind
MIT License
21 stars 1 forks source link

Incorrectly deployed Guava #4

Open paulhoule opened 10 years ago

paulhoule commented 10 years ago

2014-01-28 21:30:15,941 FATAL org.apache.hadoop.mapred.Child (main): Error running child : java.lang.NoSuchMethodError: com.google.common.base.Splitter.splitToList(Ljava/lang/CharSequence;)Ljava/util/List; at com.ontology2.telepath.projectNormalized3D.ProjectNormalized3DMapper.getYrmo(ProjectNormalized3DMapper.java:56) at com.ontology2.telepath.projectNormalized3D.ProjectNormalized3DMapper.loadNormalizationFactors(ProjectNormalized3DMapper.java:50) at com.ontology2.telepath.projectNormalized3D.ProjectNormalized3DMapper.loadNormalizationFactors(ProjectNormalized3DMapper.java:44) at com.ontology2.telepath.projectNormalized3D.ProjectNormalized3DMapper.setup(ProjectNormalized3DMapper.java:36) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132) at org.apache.hadoop.mapred.Child.main(Child.java:249)

paulhoule commented 10 years ago

I can workaround this by changing the code to not use this method, which was introduced in Guava 16. But really the misalignment should be fixed. This is a little puzzling because the mvn dependencies look OK

[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building telepath 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ telepath ---
[INFO] com.ontology2:telepath:jar:1.0-SNAPSHOT
[INFO] +- junit:junit:jar:4.11:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.springframework:spring-test:jar:4.0.0.RELEASE:test
[INFO] +- org.mockito:mockito-core:jar:1.9.5:test
[INFO] |  \- org.objenesis:objenesis:jar:1.0:test
[INFO] +- com.ontology2:bakemono:jar:t20140126:compile
[INFO] |  +- com.ontology2:centipede:jar:99.3:compile
[INFO] |  |  +- com.google.guava:guava:jar:16.0:compile
[INFO] |  |  +- org.springframework:spring-context:jar:4.0.0.RELEASE:compile
[INFO] |  |  |  +- org.springframework:spring-aop:jar:4.0.0.RELEASE:compile
[INFO] |  |  |  |  \- aopalliance:aopalliance:jar:1.0:compile
[INFO] |  |  |  +- org.springframework:spring-beans:jar:4.0.0.RELEASE:compile
[INFO] |  |  |  +- org.springframework:spring-core:jar:4.0.0.RELEASE:compile
[INFO] |  |  |  \- org.springframework:spring-expression:jar:4.0.0.RELEASE:compile
[INFO] |  |  +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] |  |  \- com.google.code.findbugs:jsr305:jar:2.0.2:compile
[INFO] |  \- org.apache.jena:jena-arq:jar:2.9.3:compile
[INFO] |     +- org.apache.jena:jena-core:jar:2.7.3:compile
[INFO] |     |  +- org.apache.jena:jena-iri:jar:0.9.3:compile
[INFO] |     |  \- xerces:xercesImpl:jar:2.10.0:compile
[INFO] |     |     \- xml-apis:xml-apis:jar:1.4.01:compile
[INFO] |     +- commons-codec:commons-codec:jar:1.4:compile
[INFO] |     +- org.apache.httpcomponents:httpclient:jar:4.1.2:compile
[INFO] |     +- org.apache.httpcomponents:httpcore:jar:4.1.3:compile
[INFO] |     +- org.slf4j:slf4j-api:jar:1.6.4:compile
[INFO] |     +- org.slf4j:slf4j-log4j12:jar:1.6.4:compile
[INFO] |     \- log4j:log4j:jar:1.2.16:compile
[INFO] \- org.apache.hadoop:hadoop-core:jar:1.0.4:provided
[INFO]    +- commons-cli:commons-cli:jar:1.2:provided
[INFO]    +- xmlenc:xmlenc:jar:0.52:provided
[INFO]    +- commons-httpclient:commons-httpclient:jar:3.0.1:provided
[INFO]    +- org.apache.commons:commons-math:jar:2.1:provided
[INFO]    +- commons-configuration:commons-configuration:jar:1.6:provided
[INFO]    |  +- commons-collections:commons-collections:jar:3.2.1:provided
[INFO]    |  +- commons-lang:commons-lang:jar:2.4:provided
[INFO]    |  +- commons-digester:commons-digester:jar:1.8:provided
[INFO]    |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:provided
[INFO]    |  \- commons-beanutils:commons-beanutils-core:jar:1.8.0:provided
[INFO]    +- commons-net:commons-net:jar:1.4.1:provided
[INFO]    +- org.mortbay.jetty:jetty:jar:6.1.26:provided
[INFO]    |  \- org.mortbay.jetty:servlet-api:jar:2.5-20081211:provided
[INFO]    +- org.mortbay.jetty:jetty-util:jar:6.1.26:provided
[INFO]    +- tomcat:jasper-runtime:jar:5.5.12:provided
[INFO]    +- tomcat:jasper-compiler:jar:5.5.12:provided
[INFO]    +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:provided
[INFO]    |  \- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:provided
[INFO]    +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:provided
[INFO]    |  \- ant:ant:jar:1.6.5:provided
[INFO]    +- commons-el:commons-el:jar:1.0:provided
[INFO]    +- net.java.dev.jets3t:jets3t:jar:0.7.1:provided
[INFO]    +- net.sf.kosmosfs:kfs:jar:0.3:provided
[INFO]    +- hsqldb:hsqldb:jar:1.8.0.10:provided
[INFO]    +- oro:oro:jar:2.0.8:provided
[INFO]    +- org.eclipse.jdt:core:jar:3.1.1:provided
[INFO]    \- org.codehaus.jackson:jackson-mapper-asl:jar:1.0.1:provided
[INFO]       \- org.codehaus.jackson:jackson-core-asl:jar:1.0.1:provided
[INFO] ------------------------------------------------------------------------
paulhoule commented 10 years ago

Here is what actually goes into the JAR when I build telepath on "Infovore A", the staging server in AWS.

[ec2-user@ip-10-194-46-122 telepath]$ unzip -l target/telepath-1.0-SNAPSHOT-job.jar | grep Splitter
     1365  01-17-2014 16:44   com/google/common/base/Splitter$1$1.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$1.class
     1592  01-17-2014 16:44   com/google/common/base/Splitter$2$1.class
     1335  01-17-2014 16:44   com/google/common/base/Splitter$2.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$3$1.class
     1543  01-17-2014 16:44   com/google/common/base/Splitter$3.class
     1350  01-17-2014 16:44   com/google/common/base/Splitter$4$1.class
     1290  01-17-2014 16:44   com/google/common/base/Splitter$4.class
     1447  01-17-2014 16:44   com/google/common/base/Splitter$5.class
     2864  01-17-2014 16:44   com/google/common/base/Splitter$MapSplitter.class
     2227  01-17-2014 16:44   com/google/common/base/Splitter$SplittingIterator.class
      431  01-17-2014 16:44   com/google/common/base/Splitter$Strategy.class
     6739  01-17-2014 16:44   com/google/common/base/Splitter.class

and it is also unzipping the same thing on my Windows development box

[ec2-user@ip-10-194-46-122 telepath]$ unzip -l target/telepath-1.0-SNAPSHOT-job.jar | grep Splitter
     1365  01-17-2014 16:44   com/google/common/base/Splitter$1$1.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$1.class
     1592  01-17-2014 16:44   com/google/common/base/Splitter$2$1.class
     1335  01-17-2014 16:44   com/google/common/base/Splitter$2.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$3$1.class
     1543  01-17-2014 16:44   com/google/common/base/Splitter$3.class
     1350  01-17-2014 16:44   com/google/common/base/Splitter$4$1.class
     1290  01-17-2014 16:44   com/google/common/base/Splitter$4.class
     1447  01-17-2014 16:44   com/google/common/base/Splitter$5.class
     2864  01-17-2014 16:44   com/google/common/base/Splitter$MapSplitter.class
     2227  01-17-2014 16:44   com/google/common/base/Splitter$SplittingIterator.class
      431  01-17-2014 16:44   com/google/common/base/Splitter$Strategy.class
     6739  01-17-2014 16:44   com/google/common/base/Splitter.class

Hail Cygwin.

paulhoule commented 10 years ago

Actually, when I compare this to what is in guava-16.0.jar, I see that this is the same

unzip -l guava-16.0.jar | grep Splitter
     1365  01-17-2014 16:44   com/google/common/base/Splitter$1$1.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$1.class
     1592  01-17-2014 16:44   com/google/common/base/Splitter$2$1.class
     1335  01-17-2014 16:44   com/google/common/base/Splitter$2.class
     1396  01-17-2014 16:44   com/google/common/base/Splitter$3$1.class
     1543  01-17-2014 16:44   com/google/common/base/Splitter$3.class
     1350  01-17-2014 16:44   com/google/common/base/Splitter$4$1.class
     1290  01-17-2014 16:44   com/google/common/base/Splitter$4.class
     1447  01-17-2014 16:44   com/google/common/base/Splitter$5.class
     2864  01-17-2014 16:44   com/google/common/base/Splitter$MapSplitter.class
     2227  01-17-2014 16:44   com/google/common/base/Splitter$SplittingIterator.class
      431  01-17-2014 16:44   com/google/common/base/Splitter$Strategy.class
     6739  01-17-2014 16:44   com/google/common/base/Splitter.class

I think the only way we could be getting a guava version before version 15 (when the above method was added) is if it was coming in with Amazon EMR...