search

fair-research / bdbag

Big Data Bag Utilities
https://fair-research.org
Apache License 2.0
49 stars 23 forks source link

support for regex fetch-filter #41

Closed alpae closed 3 years ago

alpae commented 3 years ago

I need to extract only a couple of files from a big bag, but I cannot use a simple =* or $* expression to select all the relevant files. From a quick check it looks pretty straight forward to add also a regex match, e.g. with operator =~ or similar, I think I could also provide a PR for this. would you be interested in this, or is there a reason for not including this?

mikedarcy commented 3 years ago

Sorry for the delay in responding. I think this and the associated PR are fine. I need to look into the CI failures (which don't seem to be related to this but rather setuptools_scm. Let me get those fixed and then I will merge your PR. I can't give you an exact timeframe on a PyPi release, but if you are ok pulling from Github until then, that would be great. I will try to do a point release soon.

FWIW, the reason it wasn't originally included was just to simplify the filter approach, but I always kind of regretted not including it. I think the way you have it spliced into the existing option set is good, and makes things more flexible without overly complicating things.

mikedarcy commented 3 years ago

I've merged your PR and bumped the version number in master. I'll try to get a release up to pypi/conda-forge in the next day or two.

alpae commented 3 years ago

Thanks! I can easily work on the github checkout, no rush for a release for me.

mikedarcy commented 3 years ago

This feature has been incorporated into 1.6.1.