Canceling mid-evaluation can leave a "dangling" volume, leading to leftover logs making it into the next evaluation results
Reproducer
make install + docker pull ghcr.io/symflower/eval-dev-quality:5ca853c to have the image already (there is no download progress from within the evaluation which drives me crazy)
docker volume rm evaluation-volume just to make sure no leftover volume already exists
inspect second result directory and see that there are two sub-result directories for symflower/symbolic-execution :-1:
tree evaluation-2024-10-21-12\:23\:04/
evaluation-2024-10-21-12:23:04/
├── config.json
├── evaluation.log
├── symflower_symbolic-execution # First one contains the "light" repository.
│  ├── config.json
│  ├── evaluation.csv
│  ├── evaluation.log
│  └── write-tests
│  └── symflower_symbolic-execution
│  └── golang
│  └── golang
│  ├── light
│  │  └── evaluation.log
│  └── plain
│  └── evaluation.log
└── symflower_symbolic-execution-0 # Second one contains only "plain".
├── config.json
├── evaluation.csv
├── evaluation.log
├── README.md
└── write-tests
└── symflower_symbolic-execution
└── golang
└── golang
└── plain
└── evaluation.log
Please help us keep the number of duplicated issues small.
Brief Description
Canceling mid-evaluation can leave a "dangling" volume, leading to leftover logs making it into the next evaluation results
Reproducer
make install
+docker pull ghcr.io/symflower/eval-dev-quality:5ca853c
to have the image already (there is no download progress from within the evaluation which drives me crazy)docker volume rm evaluation-volume
just to make sure no leftover volume already existseval-dev-quality evaluate --model symflower/symbolic-execution --repository golang/light --runtime docker --runtime-image ghcr.io/symflower/eval-dev-quality:5ca853c
Ctrl+C
docker volume ls
>evaluation-volume
still exists :-1:golang/plain
)eval-dev-quality evaluate --model symflower/symbolic-execution --repository golang/plain --runtime docker --runtime-image ghcr.io/symflower/eval-dev-quality:5ca853c
symflower/symbolic-execution
:-1:Logs
No response
Additional Information
No response
Version
5ca853c