search

onyx-platform / onyx-amazon-s3

Amazon S3 plugin for Onyx
1 stars 9 forks source link

Arbitrary files filter #9

Open fl00r opened 7 years ago

fl00r commented 7 years ago

Here is a proposal to add support for an arbitrary function to filter files from a bucket.

fl00r commented 7 years ago

I didn't check the code, though :). At this point, I am looking for your conceptual approval.

lbradstreet commented 7 years ago

Hi,

My apologies for the slow response. I didn't see this.

The concept is fine, though I think supplying a fn alone isn't going to be very parameterizable. Maybe if you pass the task map into the filter function that would help. The implementation also won't work because fns are not serializable. You will need to supply the fn as a keyword and then use kw->fn to resolve it.

If you want to finish the PR as I described I would be happy to merge it.

fl00r commented 7 years ago

Hi Lucas,

Thank you for your feedback.

I will finish the PR soon.