What steps will reproduce the problem?
1. Write a simple Embedded Jaql program, resulting in a M/R job, e.g.:
public static void main(String[] args) throws Exception {
JaqlQuery q = new JaqlQuery("[1,2,3,4,5]->write({type:\"hdfs\",
location:\"out.json\"});");
q.evaluate();
q.close();
}
2. Pack the program as a fatjar (i.e. containing the JAQL references directly).
3. Run the fatjar using: hadoop jar <fatjar>.
What is the expected output? What do you see instead?
The expected output would be a successul write to HDFS, instead I receive a
IllegalAccessException (see attachment).
What version of the product are you using? On what operating system?
Jaql 0.5.1 on Hadoop 0.20.2 on Ubuntu x64 10.10
Please provide any additional information below.
The error is caused by two things:
1) Jaql class "org.apache.hadoop.mapred.ExposeJobContext" calls a constroctur
of Hadoop class "org.apache.hadoop.mapred.JobContext" which is only package
visible.
2) Hadoop RunJar loads provided user classes with a different Classloader than
Hadoop classes.
3) Since "org.apache.hadoop.mapred.JobContext" and
"org.apache.hadoop.mapred.ExposeJobContext" are loaded by two different
Classloaders, the JVM does not see them as being within the same package. Thus
the call to the JobContext constructor raises an IllegalAccessException.
Original issue reported on code.google.com by mhei...@gmail.com on 25 Jan 2011 at 5:44
Original issue reported on code.google.com by
mhei...@gmail.comon 25 Jan 2011 at 5:44Attachments: