Chroma use on read-only file systems

[jasonkey]

Is it possible to use Chroma when installed on a read-only filesystem? The software appears to require write access to the software itself for API and model parameters.

Permission denied: '.../lib/python3.8/site-packages/chroma/layers/structure/params/centering_2g3n.params'

Ideally parameters and API access keys could be configured with a variable or in $HOME - is that possible?

[wujiewang]

If you cant change the permission, can you try installing directly by cloning from github to your $HOME the pip install -e .?

git clone https://github.com/generatebio/chroma.git && cd chroma && pip install -e .

[jasonkey]

Hi @wujiewang.

Yes, I've installed the application to my home directory and that is fine - it works and I am having fun playing with it.

I'd like to be able to provide chroma in an HPC environment. Typically users don't have write access to the software stack in a shared HPC setting. For chroma, it would be possible to do this if the application used parameters and configuration files from a filesystem location that could be written to by the user. Ideally that location could be defined by a variable, but it could be a home directory.

Of course every user could individually install the application themselves, provided they have an adequate python environment available. But that can be inefficient and challenging to support.

[wujiewang]

Ah, I see. Thanks for explaining the hpc usage context.

1. For the API key, I think you can specify the key_directory to somewhere you can write you with

from chroma.uitility.api import register_key 
register_key(key = 'ABCDEFG...', key_directory="/path/to/your/key") 

2. Thecentering_2g3n.params is dumped when you first call the ProteinFeatureGraph to compute reference statistics, and you can also write it to somewhere you want, but the path is hard-coded now. The relevant is code is here.

I think it is easy, you can probably just comment this line out.

Do you think you can figure something out? perhaps set environment variables $CHROMAKEY and $CHROMA_CONSTANT_PATH and let the code load these and dump the files?

[natsukium]

I'm also interested in this issue. I'm trying to package it for Nix, a package manager that also has a read-only file system.

For the API key, I think you can specify the key_directory to somewhere you can write you with

I found this argument, but the function download_from_generate called in model.py seems to expect the api key to be stored in the default directory currently. https://github.com/generatebio/chroma/blob/93071cd5a9bb547a2f1038eec3b2d750d86790e2/chroma/utility/model.py#L101-L103

It seems like a good way to allow environment variables to be set. Besides, how about changing the default save directory to XDG Base Directory, for example? XDG Base Directory is one of the specifications that defines where the application settings and data files are located in Linux, and is used by many applications as follows. https://wiki.archlinux.org/title/XDG_Base_Directory

I tried changing the following lines to xdg_config_dir and xdg_data_dir, and it works fine.

https://github.com/generatebio/chroma/blob/81f6612175f5336e1097bdb7c4339c11211a23f8/chroma/utility/api.py#L24

https://github.com/generatebio/chroma/blob/81f6612175f5336e1097bdb7c4339c11211a23f8/chroma/layers/structure/protein_graph.py#L247

[jasonkey]

Thanks Otabi, that's great. That's where I was headed as well.

Are there more hard-coded paths to fix? The API key and params paths were the first I ran into but there may be others. Have you run through any of the other examples in read-only?

[wujiewang]

@jasonkey @natsukium

Hey, will close this for now, and feel free to reopen or post new issues if you are still seeing problems. Also, consider submitting a PR and we will provide a guideline for accepting PRs.

[natsukium]

@wujiewang @aismail3-gnr8 Thanks for your fix! However, I still seem to have a problem loading the API key.

[wujiewang]

Hi, do you recommend us changing to XDG Base directory then? I still need to learn about it and test it. If you have figured out, mind submitting a PR?