search

keedio / flume-ftp-source

FTP network server is source of events for Apache-flume
80 stars 61 forks source link

Multi-threaded FTP Source #14

Closed deepakbs closed 9 years ago

deepakbs commented 9 years ago

Hi,

I am currently using the Flume FTP source to consume events from an FTP location. Since, the current FTP Flume source reads and processes one file at a time in a sequential manner it is unable to catch up with the speed at which events are arriving at the FTP location. Is there a way to speed up this by creating parallel FTP sources to consume from the same FTP location and speed this up.

Thanks Deepak

lazaromedina commented 9 years ago

Not actually because the api FTPClient of apache is blocking.