PominovaMS / denovo_benchmarks

4 stars 9 forks source link

Use environment variables for file paths so that benchmark can be run locally #20

Closed BioGeek closed 5 days ago

BioGeek commented 2 weeks ago

Currently the benchmark cannot be run locally due to hardcoded paths.

This has partially been addressed in 23dfab458a2419db08d9ec166a812cbc7c3ddab3 with the introduction of algorithms/base/constants.py.

This solution is however unsatisfactory. When multiple users modify the same algorithms/base/constants.py file to adapt it to their local environment, they could end up with version control conflicts. Each time someone clones the repo, modifies the file, pushes their changes and accidentally includes algorithms/base/constants.py, it risks overwriting or conflicting with the changes of others.

There are also still hardcoded paths in dataset_utils.py and evaluate.py.

This PR proposes a solution using a .env file. A .env file stores environment-specific configurations like file paths separately from the code. This allows each user to have their own .env file with local settings without needing to modify the codebase.

Since the .env file is ignored in version control (using .gitignore), users don't have to worry about overwriting each other's configurations. Each user maintains their own version without modifying a shared file.

This PR adds a default .env.template file to the repository with placeholders values, guiding users to create their own .env file based on their system. It uses the values that were previously hardcoded.

The README is updated and the algorithms/base/constants.py file is removed.

The environment variables are passed to the apptainer images with the --env-file .env flag.

Fixes #19.

BioGeek commented 1 week ago

Some of the variable names start with VSC (which I guess stands for HPC provider Vlaams Supercomputer Centrum), which don't make sense when running locally. So maybe also change them to more generic names like:

PominovaMS commented 4 days ago

Great solution, thank you very much for implementing it! I also updated global variables as you suggested.