search

DigitalPebble / behemoth

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.
Other
281 stars 60 forks source link

Exception when calling DistributedCache.purgeCache(job) in GATEDriver.java #25

Closed butlermh closed 13 years ago

butlermh commented 13 years ago

This may affect UIMADriver.java and SOLRIndexerJob.java as they call purgeCache(job) as well

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.filecache.DistributedCache.purgeCache(Lorg/apache/hadoop/conf/Configuration;)V at com.digitalpebble.behemoth.gate.GATEDriver.run(GATEDriver.java:112) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at com.digitalpebble.behemoth.gate.GATEDriver.main(GATEDriver.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) mark@mark-laptop:~/apps/hadoop-0.20.203.0$

butlermh commented 13 years ago

Maybe we should not be calling purgeCache at all - see

http://search-hadoop.com/m/C4Cbr1P5IE32&subj=Re+DistributedCache+purgeCache

butlermh commented 13 years ago

I have run this test against the version of behemoth in jnioche/behemoth and I get the same problem, so it was not accidentally introduced on my branch when I attempted to change to the Hadoop 0.20.2 API. So I am going to remove purgeCache in my branch.

jnioche commented 13 years ago

Fixed in commit https://github.com/jnioche/behemoth/commit/46159665eea495f6a19b75ed381df396698a1433