search

apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.47k stars 3.7k forks source link

druid 0.12.1 cant load gplcompression #8453

Closed wsjz closed 5 years ago

wsjz commented 5 years ago

I have no idea for this problem on EMR clusters, I have nearly tried all method .

2019-09-03T03:21:46,819 INFO [task-runner-0-priority-0] org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at nge480390-master-instance-4qz0nkxz.novalocal/192.168.11.157:8050 2019-09-03T03:21:47,362 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobResourceUploader - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 2019-09-03T03:21:47,435 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 2019-09-03T03:21:49,144 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2019-09-03T03:21:49,216 ERROR [task-runner-0-priority-0] com.hadoop.compression.lzo.GPLNativeCodeLoader - Could not load native gpl library java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path

wsjz commented 5 years ago

i have add all lib from /hadoop/lib to /druid/lib/, and make sure java.library.path have these libs

I think it affected by this?

WARN JNDI lookup class is not available because this JRE does not support JNDI. JNDI string lookups will not be available, continuing configuration. java.lang.ClassCastException: Cannot cast org.apache.logging.log4j.core.lookup.JndiLookup to org.apache.logging.log4j.core.lookup.StrLookup

wsjz commented 5 years ago

and this

2019-09-03T05:15:14,592 ERROR [task-runner-0-priority-0] com.hadoop.compression.lzo.LzoCodec - Cannot load native-lzo without native-hadoop

vogievetsky commented 5 years ago

Is this an issue on newer Druids like 0.15.1?

wsjz commented 5 years ago

@vogievetsky I have solved it ，add LD_LIBRARY_PATH to druid-env.xml for ambari druid stack(HDP3.1 stack).