[FEATURE REQUEST] add parameter --temp for temporary files path

Louis-MG commented 1 month ago

The need

Diamond can generate a lot of temporary files. The default /tmp can get full on clusters.

The solution

Add a parameter --temp to the command anvi-pan-genome, maybe others too.

Beneficiaries

Anyone working with anvio.

meren commented 1 month ago

Hi @Louis-MG, anvi'o already has a way to help the user to explicitly define a path for temporary files. Can you please try this and let us know if this solves your problem?

# set the TMPDIR environmental variable
export TMPDIR="/path/of/your/choosing/

# run your command in the same terminal session
anvi-pan-genome [...]

Louis-MG commented 1 month ago

It did work, thank you ! But I feel like a flag would be more straightforward. Feels a bit overkill too.

meren commented 1 month ago

We certainly can do that :)

I will mark this as a small project in case someone wishes to add this feature into the codebase.

Louis-MG commented 1 month ago

I'll try to contribute when i find the time !

meren commented 1 month ago

Some quick notes for anyone who wishes to work on this :)

All temporary directory operations are handled in here: https://github.com/merenlab/anvio/blob/master/anvio/filesnpaths.py -- get_temp_directory_path is the function that needs to change if there is a user-provided directory (tempfile.mkdtemp() reads environmental variables or creates a directory by itself).
We have a bunch of 'community' params that are available in ALL anvi'o programs, such as --debug, they are listed here: https://github.com/merenlab/anvio/blob/master/anvio/__init__.py#L63 (and how they are used can be seen in the codebase).
One critical consideration would be to NOT delete directories if users manually sets a directory via the new --temp-directory parameter and the programmer passes true for just_the_path in the header of get_temp_directory_path.

merenlab / anvio