frej / fast-export

A mercurial to git converter using git-fast-import
http://repo.or.cz/w/fast-export.git
808 stars 255 forks source link

Added ability for file filter plugins to discard files #262

Closed Dakkaron closed 3 years ago

Dakkaron commented 3 years ago

This pull request allows file data filter plugins to discard files.

For example, if you use this plugin:

def build_filter(args):
    return Filter(args)

class Filter:
    def __init__(self, args):
        self.whitelistedPrefixes = args.split(",")

    def file_data_filter(self,file_data):
        filename = file_data['filename']
        file_data["drop_file"] = not any([ filename.startswith(x) for x in self.whitelistedPrefixes ])

and call it with the following parameters: --plugin file_prefix_whitelist='src/'

only files in the sub-folder src/ will be included.

This might create some empty commits, but that doesn't cause any issues.

Dakkaron commented 3 years ago

I hope this is better now

Dakkaron commented 3 years ago

filter_by_path_prefix would be a useful term, but this project uses the word filter for a concept that really doesn't have much to do with filtering, but instead uses it to mean something like modifier. So naming this a filter would be rather misleading.

The issue with the word whitelist is that it really doesn't have anything to do with racism, but is instead a really descriptive term for exactly what is meant here.

Tbh, this pull request has two lines of useful code. I could have just made an issue with a feature request, now I'm slogging through a racism debate for some example code. If you want to, lets just delete the example and people can figure out how to use that thing themselves.

frej commented 3 years ago

filter_by_path_prefix would be a useful term, but this project uses the word filter for a concept that really doesn't have much to do with filtering, but instead uses it to mean something like modifier. So naming this a filter would be rather misleading.

Good point, that's true, the methods in the plugin API should really be renamed, but I don't think anyone will be confused. At least I haven't received any issues on that aspect of plugin use. Git calls the corresponding functionality in filter-branch (which could be an alternative for one-off conversions) a subdirectory-filter (it also has index-filter).

The issue with the word whitelist is that it really doesn't have anything to do with racism, but is instead a really descriptive term for exactly what is meant here.

This is the kind of debate I wanted to avoid by my suggestion to not use a word that I know is a trigger for certain activists.

Tbh, this pull request has two lines of useful code. I could have just made an issue with a feature request, now I'm slogging through a racism debate for some example code. If you want to, lets just delete the example and people can figure out how to use that thing themselves.

It's up to you to decide on what to do, either address the review comment in question or drop the plugin example.

Dakkaron commented 3 years ago

I added the changes you wanted.

frej commented 3 years ago

No activity in more than 90-days, closing.