ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

Unable to change setting of data_path in Synda #193

Closed xlevine closed 2 years ago

xlevine commented 2 years ago

Hello,

I am in the process of setting up Synda and I am encountering an issue: while I can easily install data in the data folder setup by default ( $ST_HOME/.synda/data/ ), I am unable to change this setting to have the data installed in another folder. I have tried to change this in $ST_HOME/.synda/conf/std.conf by changing the data_path option (my ST_HOME=/cluster/projects/nn8002k/.synda ):

----- In std.conf:

[daemon] user = group =

[module] download = true

[log] verbosity_level = info scheduler_profiling = 0

[core] security_dir_mode = tmpuid metadata_server_type = esgf_search_api selection_path = /cluster/projects/nn8002k/.synda/selection default_path = /cluster/projects/nn8002k/.synda/conf/default data_path = /trd-project4/NS8002K/xale/ESGF db_path = /cluster/projects/nn8002k/.synda/db sandbox_path = /cluster/projects/nn8002k/.synda/sandbox

..... Whatever path I set up for data_path is not taken into account by Synda; instead the default data path ( .synda/data ) is still used by synda install. Is there a way to change data folder path to be different than its default setting, and if so how to do it?

Many thanks for your help, any guidance would be much appreciated.

Best regards, Xavier.

hot007 commented 2 years ago

Hi Xavier,

I also found this, this might not be a helpful response but I created (my equivalent of) /cluster/projects/nn8002k/.synda/data that was a symlink to my intended data_path.

-Claire

pjournou-ipsl commented 2 years ago

To put it simply, we know that the outsourcing of the configuration offered by Synda is unfortunately not yet satisfactory. The use case you detail proves it.

Probably for historical reasons... The first choice that was made to locate the Synda workspace was based on a Linux environment variable (ST_HOME). It was a simple solution, but obviously too rigid. Customizing the paths in the user preferences file is a much better solution, but it is effectively ignored at the moment.

Todo... We will complete this work as quickly as possible.

How to simplify your life while waiting ... For now, Claire's suggestion is the only way to point your data to the desired directory.

Sorry for the inconvenience. Best regards, Patrice

xlevine commented 2 years ago

Hi @hot007 and @pjournou-ipsl

Thank you so much for your answer. I now understand why I couldn't use the data_path option. Instead, I will use the soft link option described by Claire.

One reason I wanted to change the data folder path was to do a synda list or synda stat on an already installed database (installed using synda but by a different user), and compare files found with my own search criterion (like that I would only have to download missing files in my own data folder, without having to download data already installed in that other database). Anyway, I just wanted to give you some context.

Thanks again for your quick help, I really appreciate.

Best, Xavier.

pjournou-ipsl commented 2 years ago

Let inform you that in the future 3.4 release (2021), synda will take into account the paths, as customized into the sdf.conf file.

xlevine commented 2 years ago

Excellent news, thank you for keeping me posted!