How do I use my own filter logic

keedio / flume-ftp-source

FTP network server is source of events for Apache-flume

80 stars 61 forks source link

How do I use my own filter logic #33

Closed CrazyBeeline closed 6 years ago

CrazyBeeline commented 6 years ago

hi ！

Some of the files in the compressed file are a description of the file I want to exclude these description files How do I do it

lazaromedina commented 6 years ago

Hi CrazyBeeline, i am no sure if i understand your issue. To be able to help you, could you please?:

add more deatiled information about what you are trying to do.
add a log of the problem, error, etc.

For filtering files to process or not you can try with property file.pattern. It is a java regula expression.

best, Luis

CrazyBeeline commented 6 years ago

I'm sorry for such a long time to reply to the news!!

I mean, there are some data files in the zip package, and some XML files are used to describe the data files. My requirement is to send only the data files, and the XML files are not transmitted.

lazaromedina commented 6 years ago

hi CrazyBeeline,

you have to use Java Regex, with a negative lookahead you can rule out specific extensions.
anohter option is to move out from incoming flume what you don want to process.

I m closing issue because is nor related with Flume source FTP , but with Java regular expressions.

best, Luis