Configure a work directory for output files

[mcduffie]

It would be nice if all the output files could be placed in a single working directory that may live on a different filesystem than the source directory. Right now I have to make symbolic links to each output directory from the source directory. I think this would involve an additional configuration variable which sets the prefix for all the other output directories in the configuration.

[pernak18]

i'm a little confused. my intent was to use the outdir field in ABSCO_config.ini for what you're requesting. i.e., all netCDFs that are generated are supposed to be saved in ${ROOT}/nc_ABSCO if the outdir field is not changed. i have done this for a few species (computed in separate runs), and my output looks like:

% pwd
/rd47/scratch/JPL/ABSCO/git

% ls -d */
AER_Line_File/  LBLRTM/    LNFL/       nc_ABSCO/  PT_grid/      TAPE3_dir/
common/         LBL_Runs/  LNFL_Runs/  obsolete/  __pycache__/  VMR/

% ls nc_ABSCO/
H2CO_27000-27100_v0.0_init.nc  O2_00000-00150_v0.0_init.nc
NO2_27000-27100_v0.0_init.nc

and you should be able to use an absolute path for outdir such that you can write to other file systems, assuming the FS is mounted. i just did a test and was able to write to a NAS directory rather than my local drive.

can you not replicate this behavior?

[mcduffie]

My issue is not for the output but for the work/temporary files. The files produced into LBL_runs, LNFL_Runs, TAPE3_dir, TAPE5_dir, etc. I want to keep those all together instead of in the source directory.

[pernak18]

instead of symbolic links, have you tried full paths (to your other file system) for the LBL_runs, LNFL_Runs, TAPE3_dir, and TAPE5_dir fields in the configuration file? i did want to give the user the flexibility to have those guys wherever they wanted, rather than follow my convention of having them in the source directory.

[mcduffie]

I tried full paths but the code seemed to be prefixing the directory I specified with the directory of the source code.

So you would have two full paths concatenated together.

[pernak18]

ok, sorry. that was not the intent -- that's another bug i'll have to fix

[pernak18]

@mcduffie i think i have addressed this request. let me know if anything is awry

[mcduffie]

Please also add code to create the directories under top_dir if they don't exist. That way the user does not need to know the directory structure that needs to exist a priori.

Also please remember to document topdir in the README.

[mcduffie]

There seems to be some inconsistent usage of topdir in the code. For the LNFL runs it appears in ABSCO_computer.py that runLNFL still chdirs to the relative config.lnfl_run_dir directory instead of os.path.join(config.top_dir, config.lnfl_run_dir) as I would expect.

[mcduffie]

Also it appears now that ptfile and vmrfile need to be full paths due to the chdir done to topdir.