search

HadoopGenomics / Hadoop-BAM

Hadoop-BAM is a Java library for the manipulation of files in common bioinformatics formats using the Hadoop MapReduce framework
MIT License
69 stars 52 forks source link

Hadoop BAM : Class not found Error. #14

Open shaliniravi opened 9 years ago

shaliniravi commented 9 years ago

Currently we are migrating traditional genome analysis to Hadoop and also for learning purpose. We are using bsmap for bisulfide methylation extraction (methratio.py with input = .ba file and reerence = .fa file). I tried to use hadoop bam for this purpose but I couldn't figure what I am doing wrong here:

  1. I loaded the .bam file to hdfs

but when I try to execute:

hadoop jar /usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.5.1.jar -libjars hadoop-bam-7.0.0.jar,htsjdk-1.118.jar -inputformat org.seqdoop.hadoop_bam.BAMInputFormat -file ./methratio.py -file '../fadata/Genome.fa' -mapper methratio.py -input ./wgEncodeSydhRnaSeqK562Ifna6hPolyaAln.bam -output ./outfile

I am getting :

Exception in thread "main" java.lang.NoClassDefFoundError: htsjdk/samtools/seekablestream/SeekableStream at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1986) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1951) at org.apache.hadoop.streaming.StreamUtil.goodClassOrNull(StreamUtil.java:51) at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:784) at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:128) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.ClassNotFoundException: htsjdk.samtools.seekablestream.SeekableStream at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 15 more

I would really appreciate the help.

AndreSchumacher commented 9 years ago

I'm not very familiar with the way Hadoop-streaming handles input formats. Could you try passing the *-jar-with-dependencies.jar with the -libjars argument? Also then it would be best to drop the htsjdk-1.118.jar from there.