Several files modified to allow for os-specific commands and file/directory references. In python, this was done through consistent use of the os.path commands, in GAMS it was through consistent use of %ds% global, and in R it was through file.path function calls. Several file/call flow modifications are included here with more details in the list below.
The Mac tests completed for this branch used:
macOS Mojave v10.14.6
Anaconda 4.8.1 with Python 3.7.4
R 3.6.2 (installed alongside Anaconda)
GAMS 24.7
The Linux tests completed for this branch used:
RedHat (should work on CentOS too)
Anaconda 4.7.12 with Python 3.8.0
R 3.6.1 (installed alongside Anaconda)
GAMS 24.7
A main difference in running the model, relative to windows, is that the cases do not pop up in new command prompts and instead run in the background. Said differently, on Mac machines the model instances run quietly with minimal status updates printed to the screen and the user should investigate the cases' gamslog.txt files to observe progress. The reason for this is there is no succinct means to launch a new terminal on a Mac as compared to the 'start /wait ...' command available in Windows. Therefore, I get around this by creating a python subprocess.Popen instance that runs in the background.
Instructions for Linux (RedHat or CentOS):
The commands below can be added to your .bashrc file or run each time you want to run ReEDS (good for HPC environments)
Check how the HOME variable is declared and adjust if necessary - this is used to tell Anaconda where to store its ReEDS environment
export HOME="/your/home/directory/"
Export where you'd like to store your R packages:
export R_LIBS_USER="/my/RLib/directory"
Add the GAMS system directory to $PATH:
export PATH=$PATH:/your/gams/dir/
--
The next few commands should either be stored in a script and called when you want to run ReEDS or simply run line-by-line
From the cloned repo directory the following three lines 1) create the 'reeds' environment in $HOME/.conda/reeds 2) activate that environment then 3) check for all R packages and installs those that are missing - note that gdxrrw is not on CRAN and so needs to be installed from the tgz file
conda env create -f environment.yml
conda activate reeds
rscript input_processing/R/packagesetup.R
Keep in mind that if you install the 'reeds' conda environment and want to reinstall an updated version, you'll need to add the --force argument to the end of the first line (conda env create..) above or change the assignment in environment.yml
If setup goes well, you can then 'python runbatch.py' and run cases as you would any other machine - note it does not pop open command prompts like on the Windows version and instead runs in the background with output being written to the cases gamslog.txt file
A few caveats (several to be added to the ReadMe and User Guide):
The user should either a) use the GAMS terminal application that is installed alongside GAMS -or- permanently add their GAMS system directory to the PATH environment variable (similar to windows, e.g. export PATH="gams/sysdir":$PATH )
The solver is now specified in cases.csv - allowing the user to quickly choose between cplex/gurobi/...; this was previously os-specific given our usage of gurobi on the HPC
The user might need to specify the R_LIBS_USER environment variable similar to the adjustment of the path environment variable (e.g. export R_LIBS_USER=/my/R/library/directory/) - this is typically only true if the user does not have admin rights
The installation of the gdxrrw package in packagesetup.r is now os-specific and uses gdxrrw_1.0.5.tgz as the installation file on mac/linux machines (the installation file is being pushed with this commit)
reflow bash/command prompt output is now written to gamslog.txt (see changes to d_callreflow) - note the logoption is set to '2' for Mac/Linux so that it does not print to the screen during the call to runbatch.py, keeping the screen clean of anything besides what is purposefully printed by python
Small change to LDC_prep.py that did not cause issues in Windows but caused several issues on a Mac - spoke to WF who did not see an issue with this change
No longer creating the outputs/variabilityFiles folder in d_solveprep.gms as this was already done in runbatch.py
reflowbatch.py calls to the bash mimic the adjustments made in runbatch.py (subprocess.Popen...)
Adding notes to runbatch.py to indicate that for the intertemporal and window cases you'll need to specify GAMSDir in the setupEnvironment() function of runbatch.py
Removing calls to bokeh at the end of e_reportdump and having them written to the call...sh/bat file via runbatch.py
(unrelated patch) - Scaling emit_tax by emit_scale
(cautionary note) - We will soon need to update/adjust our usage of the reshape2 package in R -- some of the commands are to be deprecated soon and will begin to cause errors and not just warnings
Enabling Mac and Linux Compatibility
Several files modified to allow for os-specific commands and file/directory references. In python, this was done through consistent use of the os.path commands, in GAMS it was through consistent use of %ds% global, and in R it was through file.path function calls. Several file/call flow modifications are included here with more details in the list below.
The Mac tests completed for this branch used: macOS Mojave v10.14.6 Anaconda 4.8.1 with Python 3.7.4 R 3.6.2 (installed alongside Anaconda) GAMS 24.7
The Linux tests completed for this branch used: RedHat (should work on CentOS too) Anaconda 4.7.12 with Python 3.8.0 R 3.6.1 (installed alongside Anaconda) GAMS 24.7
A main difference in running the model, relative to windows, is that the cases do not pop up in new command prompts and instead run in the background. Said differently, on Mac machines the model instances run quietly with minimal status updates printed to the screen and the user should investigate the cases' gamslog.txt files to observe progress. The reason for this is there is no succinct means to launch a new terminal on a Mac as compared to the 'start /wait ...' command available in Windows. Therefore, I get around this by creating a python subprocess.Popen instance that runs in the background.
Instructions for Linux (RedHat or CentOS):
The commands below can be added to your .bashrc file or run each time you want to run ReEDS (good for HPC environments)
Check how the HOME variable is declared and adjust if necessary - this is used to tell Anaconda where to store its ReEDS environment
export HOME="/your/home/directory/"
Export where you'd like to store your R packages: export R_LIBS_USER="/my/RLib/directory"
Add the GAMS system directory to $PATH: export PATH=$PATH:/your/gams/dir/
--
The next few commands should either be stored in a script and called when you want to run ReEDS or simply run line-by-line
From the cloned repo directory the following three lines 1) create the 'reeds' environment in $HOME/.conda/reeds 2) activate that environment then 3) check for all R packages and installs those that are missing - note that gdxrrw is not on CRAN and so needs to be installed from the tgz file conda env create -f environment.yml conda activate reeds rscript input_processing/R/packagesetup.R
Keep in mind that if you install the 'reeds' conda environment and want to reinstall an updated version, you'll need to add the --force argument to the end of the first line (conda env create..) above or change the assignment in environment.yml
If setup goes well, you can then 'python runbatch.py' and run cases as you would any other machine - note it does not pop open command prompts like on the Windows version and instead runs in the background with output being written to the cases gamslog.txt file
A few caveats (several to be added to the ReadMe and User Guide):