Wonderdog with Hadoop2/YARN

infochimps-labs / wonderdog

Bulk loading for elastic search

http://infochimps.com

Apache License 2.0

186 stars 56 forks source link

Wonderdog with Hadoop2/YARN #20

Open murdockcrc opened 10 years ago

murdockcrc commented 10 years ago

I compiled Pig to use Hadoop2.2.0 and installed ElasticSearch and Wonderdog. Running a Pig job in this setup fails with the following error:

"java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)"

I was wondering if there is way to build Wonderdog for it to work with Hadoop2/YARN.

Thanks

margaretlange commented 9 years ago

I was able to get rid of this kind of error by recreating the Wonderdog project with the dependency in the pom.xml file updated. Change the artifact id to "hadoop-client" and the version to "2.2.0."