feat: add bucket_prefix option to limit lists to specific directory

jalonsoa commented 3 years ago

Release notes

What does this PR do?

Add a new parameter to allow select the directory to list inside the bucket.

Why is it important/What is the impact to the user?

If you have a bucket with different files from different proyects and you only want files from one directory you can only filter with exclude_options. If the bucket have many files you get the full file list each iteration. This can cause a waste of resources (bandwidth, for example).

If we add this prefix option, you get only the file list on your directory and not the complete list of files on the bucket.

Checklist

[X ] My code follows the style guidelines of this project
[ ] I have commented my code, particularly in hard-to-understand areas
[X ] I have made corresponding changes to the documentation
[ ] I have made corresponding change to the default configuration files (and/or docker env variables)
[ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

How to test this PR locally

Related issues

Use cases

If you share the bucket with another applications or you need get files only on specific sub directory.

Screenshots

Logs

cla-checker-service[bot] commented 3 years ago

💚 CLA has been signed

jalonsoa commented 3 years ago

/retest

logstash-plugins / logstash-input-google_cloud_storage