Certora / gambit

Using mutations to improve specs and test suites
MIT License
165 stars 8 forks source link

Improve --outdir argument behavior #23

Open Sta1400 opened 1 year ago

Sta1400 commented 1 year ago

The current behavior of the --outdir argument in the Gambit tool presents some inconveniences when generating mutants using the command gambit mutate --filename <path1> --outdir <path2>. The mutants are currently stored under the directory structure <path2>/mutants/[1...Nb_Max]/<path1>/<source_contract_name>.sol. This issue proposes a more straightforward and organized directory structure for storing mutants, which would be <path2>/mutants-<source_contract_name>/mutant[1...Nb_Max].sol

Motivation :

The current directory structure for storing mutants can become cumbersome to navigate, especially when dealing with a large number of mutants. By revising the directory structure as suggested, all mutants would be consolidated in a single location, simplifying management and improving overall usability.

Proposed Directory Structure:

The suggested modification to the --outdir behavior would create the following directory structure:

<path2>/mutants-<source_contract_name>/mutant[1...Nb_Max].sol

Where:

path2: The provided output directory path. : The name of the source contract, extracted from . mutant[1...Nb_Max].sol: The individual mutant files, numbered from 1 to the maximum number of mutants generated. Benefits:

Simplified organization: All mutants for a particular source contract would be grouped together under the mutants- directory. Improved accessibility: Users would find it easier to locate and manage mutants, as they would be stored in a single, consistent location. Enhanced usability: The proposed directory structure eliminates the need to navigate multiple levels of directories, streamlining the process of accessing and analyzing mutants.

BenTheKush commented 1 year ago

Hey @Sta1400, yeah there is a tradeoff here. The reason I landed on this layout structure is that it makes clear the exact location in the project of the file that was mutated. A mutant isn't just a change in code, it's also the location in the project that the change takes place.

What is your use case? Do you want to view the mutants directly? There is the gambit summary that will summarize a given mutant:

gambit summary --mids 1

This will output something like: Screen Shot 2023-08-02 at 3 52 57 PM