Open xander-wang opened 5 years ago
@xander-wang Try putting all the packages into the classpath of Flume
package your jar with flume-1.8.0-cdh6.0.1,not flume-1.8.0,with maven:
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-core</artifactId>
<version>1.8.0-cdh6.0.1</version>
<scope>provided</scope>
</dependency>
in flume-1.8.0-cdh6.0.1, org.apache.flume.Context.getSubProperties return java.util.Map, not com.google.common.collect.ImmutableMap.
public Map<String, String> getSubProperties(String prefix) { Preconditions.checkArgument(prefix.endsWith("."), "The given prefix does not end with a period (" + prefix + ")"); Map<String, String> result = Maps.newHashMap(); synchronized (parameters) { for (String key : parameters.keySet()) { if (key.startsWith(prefix)) { String name = key.substring(prefix.length()); result.put(name, parameters.get(key)); } } } return ImmutableMap.copyOf(result); }
Trying to implement [MySQL->HBase, Kafka] in CDH6.0.1 Flume config file:
Got following once init the flume service:
I tried to recompile the source code or use other 'sql-source' releases, all in vain. It seems to be a DB connection problem after I checked the specific code, and I'm not sure about whether the method 'org.apache.flume.Context.getSubProperties' exists or not in flume1.8. Or maybe there's error in the config file, which I'm not sure as well... Just a beginner, help is appreciated!