Log file anonymizer

[mikebentley15]

Feature Request

Description:

To share some of the generated FLiT data from some collaborators with others that don't have access to the code in question can be tricky. The code worked on may be protected. You may be able to share it if some of the protected fields become anonymized somehow. Some of the fields that may be protected are:

• source filename
• source filepath
• object filename (has the source filename within it)
• test name (if tests are closed-source)
• function name before and after C++ demangling

Suggested change:

Create a new FLiT subcommand called flit disguise that generates an anonymizer mapping and can use it to convert any text file to an anonymous version of the same file.

The anonymizer map is a simple CSV file with three columns:

1. Type (i.e., field type)
2. Value
3. Replacement or disguise

How to generate?

• Can be made from Makefile in bash or python
• Can be made from flit_disguise.py directly

The main problem for function and symbol names is that we need at least one compiled version available.

• Let's use gtrun for it with its associated object files
• Let's have a Makefile target called disguises.csv and depends on gtrun and all of its object files
  • Calls flit disguise --generate --output disguises.csv
  • Calling flit disguise without the --generate flag and without --disguise-map specified will automatically call make disguise.csv which then calls flit disguise again if disguise.csv needs to be regenerated (and will also ensure that gtrun is already built)

Usage: flit disguise [options] <file>

Command-line options:

• -g, --generate: generates the disguise map csv
• -o, --output: output to a specified file instead of to standard output (which is the default)
• -m, --disguise-map: specify to use this disguise map instead of the default disguise.csv. This will not check to see if the disguise map needs to be regenerated.
• -u, --undo: undo the disguising, uncovering the original content (also known as deanonymizing).
• --fields: a comma-separated list of fields you want to disguise. This will not effect the generated disguise map, all fields will be present there. It will just impact the disguising and the undo operations.
• <file>: the file to disguise or undo.

[mikebentley15]

I guess I forgot to mention how it would be implemented.

The disguises.csv file would be generated using the same information that flit bisect uses to enumerate its search. The replacement names could simply be <field-type>-<number>, for example filename-001.

For doing the disguising and the undo operation, I'm thinking simple search and replace using the python re module. That would make it capable of handling any file type, not just ones specific log files. Care must be taken to ensure a partial match is not replaced. For instance, if we have a function named sub(), we won't replace submarine with func-001marine

[mikebentley15]

It may also be good to specify - instead of a file to specify that we want to read from stdin. Or we could have the default to be read from stdin. Just a thought.

[ganeshutah]

All these comments on implementing disguise are right on target!

Ganesh

On Sat, Jul 11, 2020 at 4:13 PM Michael Bentley notifications@github.com wrote:

It may also be good to specify - instead of a file to specify that we want to read from stdin. Or we could have the default to be read from stdin. Just a thought.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/PRUNERS/FLiT/issues/328#issuecomment-657137941, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGKECQIKA3NP45ADVHLYNLR3DPYDANCNFSM4OXPSFFQ .