Closed feature09 closed 6 years ago
Hi, i'm exec stream data from mysql table into HDFS,but it's error.
my flume.conf: agent.channels = ch1 agent.sources = sql-source agent.sinks = HDFS
agent.channels.ch1.type = memory
agent.sources.sql-source.type = org.keedio.flume.source.SQLSource
agent.sources.sql-source.connection.url = jdbc:mysql://192.168.44.21:3306/test agent.sources.sql-source.user = root agent.sources.sql-source.password = 123456 agent.sources.sql-source.table = flumelog agent.sources.sql-source.columns.to.select = *
agent.sources.sql-source.channels = ch1 agent.sources.sql-source.incremental.column.name = id agent.sources.sql-source.incremental.value = 0
agent.sources.sql-source.run.query.delay=5000
agent.sources.sql-source.status.file.path = /root/flume-test agent.sources.sql-source.status.file.name = sql-source.status agent.sinks.HDFS.channel = ch1 agent.sinks.HDFS.type = hdfs agent.sinks.HDFS.hdfs.path = hdfs://192.168.44.21/flume/mysql-flume agent.sinks.HDFS.hdfs.fileType = DataStream agent.sinks.HDFS.hdfs.writeFormat = Text agent.sinks.HDFS.hdfs.rollSize = 268435456 agent.sinks.HDFS.hdfs.rollInterval = 0 agent.sinks.HDFS.hdfs.rollCount = 0
==================================================== executing command: flume-ng agent --conf /home/apps/flume/conf/ --conf-file mysql-flume --name agent -Dflume.root.logger=INFO,console
==================================================== the part of log in console is:
2018-05-21 15:31:19,999 (PollableSourceRunner-SQLSource-sql-source) [WARN - org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:144)] SQL Error: 1146, SQLState: 42S02 2018-05-21 15:31:19,999 (PollableSourceRunner-SQLSource-sql-source) [ERROR - org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:146)] Table 'test.flumelog' doesn't exist Exception in thread "PollableSourceRunner-SQLSource-sql-source" java.lang.AbstractMethodError: org.keedio.flume.source.SQLSource.getMaxBackOffSleepInterval()J at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:150) at java.lang.Thread.run(Thread.java:745)
Hi, i'm exec stream data from mysql table into HDFS,but it's error.
my flume.conf: agent.channels = ch1 agent.sources = sql-source agent.sinks = HDFS
agent.channels.ch1.type = memory
agent.sources.sql-source.type = org.keedio.flume.source.SQLSource
agent.sources.sql-source.connection.url = jdbc:mysql://192.168.44.21:3306/test
agent.sources.sql-source.user = root
agent.sources.sql-source.password = 123456
agent.sources.sql-source.table = flumelog
agent.sources.sql-source.columns.to.select = *
agent.sources.sql-source.channels = ch1 agent.sources.sql-source.incremental.column.name = id
agent.sources.sql-source.incremental.value = 0
agent.sources.sql-source.run.query.delay=5000
agent.sources.sql-source.status.file.path = /root/flume-test agent.sources.sql-source.status.file.name = sql-source.status
agent.sinks.HDFS.channel = ch1
agent.sinks.HDFS.type = hdfs
agent.sinks.HDFS.hdfs.path = hdfs://192.168.44.21/flume/mysql-flume agent.sinks.HDFS.hdfs.fileType = DataStream
agent.sinks.HDFS.hdfs.writeFormat = Text
agent.sinks.HDFS.hdfs.rollSize = 268435456
agent.sinks.HDFS.hdfs.rollInterval = 0
agent.sinks.HDFS.hdfs.rollCount = 0
==================================================== executing command: flume-ng agent --conf /home/apps/flume/conf/ --conf-file mysql-flume --name agent -Dflume.root.logger=INFO,console
==================================================== the part of log in console is:
2018-05-21 15:31:19,999 (PollableSourceRunner-SQLSource-sql-source) [WARN - org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:144)] SQL Error: 1146, SQLState: 42S02 2018-05-21 15:31:19,999 (PollableSourceRunner-SQLSource-sql-source) [ERROR - org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:146)] Table 'test.flumelog' doesn't exist Exception in thread "PollableSourceRunner-SQLSource-sql-source" java.lang.AbstractMethodError: org.keedio.flume.source.SQLSource.getMaxBackOffSleepInterval()J at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:150) at java.lang.Thread.run(Thread.java:745)