Verify case study results are reproducible with final DISPATCHES release candidate

adowling2 commented 1 year ago

Goal

Verify the key results in each case study report are reproducible with the final DISPATCHES release candidate

Tasklist

RE case study (generic)

[ ] @dguittet price taker for RE case study (#208)
[x] @Xinhe-Chen train surrogates for RE case study*
[ ] @Xinhe-Chen solve surrogates embedded optimization for RE case study* (#208)
[ ] @dguittet @Xinhe-Chen Prescient verification simulation (#208)

NE case study (generic) (#210)

[x] @radhakrishnatg price-taker for NE case study
[x] @Xinhe-Chen train surrogates for NE case study*
[x] @Xinhe-Chen @radhakrishnatg solve surrogates embedded optimization for NE case study*
[x] @radhakrishnatg @Xinhe-Chen Verification

*TensorFlow will likely be pinned to 2.12 (or 2.10)

FE case study (PR #209 )

[x] @esrawli GDP optimization
[x] @nareshsusarla multiperiod optimization

Others

[x] @dguittet RE collaborative case study (#215)
[ ] @radhakrishnatg NE collaborative case study

Instructions

Set up environment to be used for verification (see below)
Post to this issue details about which files you ran for verification. We will use comments on this issue as documentation.
Once the verification is done, check off the task above.

Set up environment for verification

[ ] Use a new directory with a freshly cloned repository (i.e. not your existing local clone used for development)
[ ] Create a new environment

mkdir dispatches-verify
cd dispatches-verify
git clone https://github.com/gmlc-dispatches/dispatches
cd dispatches
git checkout 1.3.0rc0
conda create --yes -n dispatches-verify-1.3.0rc0 python=3.9
conda activate dispatches-verify-1.3.0rc0
pip install -r requirements-dev.txt
idaes get-extensions --verbose
pip show pyomo idaes-pse dispatches

The output of the last two commands should look something like this:

(dispatches-verify-1.3.0rc0) C:\Users\lbian\proj\dispatches-verify\dispatches>idaes get-extensions --verbose
Getting files...
DEBUG   idaes.commands.util.download_bin: Downloading binaries for windows-x86_64
DEBUG   idaes.commands.util.download_bin: Downloading binaries from https://github.com/IDAES/idaes-ext/releases/download/3.2.0
DEBUG   idaes.commands.util.download_bin: Getting checksum file https://raw.githubusercontent.com/IDAES/idaes-ext/main/releases/sha256sum_3.2.0.txt
DEBUG   idaes.commands.util.download_bin: Getting lib from: https://github.com/IDAES/idaes-ext/releases/download/3.2.0/idaes-lib-windows-x86_64.tar.gz
DEBUG   idaes.commands.util.download_bin: Saving solvers to: C:\Users\lbian\AppData\Local\idaes\bin\idaes-lib-windows-x86_64.tar.gz
DEBUG   idaes.commands.util.download_bin: Getting solvers from: https://github.com/IDAES/idaes-ext/releases/download/3.2.0/idaes-solvers-windows-x86_64.tar.gz
DEBUG   idaes.commands.util.download_bin: Saving solvers to: C:\Users\lbian\AppData\Local\idaes\bin\idaes-solvers-windows-x86_64.tar.gz
DEBUG   idaes.commands.util.download_bin: lib Hash ba4b18e133e9ecf99e1d496396682aaaccc2c3e11ddc81ca3427e5fddf323b3a
DEBUG   idaes.commands.util.download_bin: solvers Hash 7d474bbf7ee7d0d525d96c3d37f65a84bc205baf722456a7289b7676a556a467
DEBUG   idaes.commands.util.download_bin: Extracting files in C:\Users\lbian\AppData\Local\idaes\bin\idaes-lib-windows-x86_64.tar.gz to C:\Users\lbian\AppData\Local\idaes\bin
DEBUG   idaes.commands.util.download_bin: Extracting files in C:\Users\lbian\AppData\Local\idaes\bin\idaes-solvers-windows-x86_64.tar.gz to C:\Users\lbian\AppData\Local\idaes\bin
Done
---------------------------------------------------
IDAES Extensions Build Versions
===================================================
Solvers:  v3.2.0 20230203-1520 windows-x86_64
Library:  v3.2.0 20230203-1552 windows-x86_64
===================================================

(dispatches-verify-1.3.0rc0) C:\Users\lbian\proj\dispatches-verify\dispatches>pip show pyomo idaes-pse dispatches
Name: Pyomo
Version: 6.5.0
Summary: Pyomo: Python Optimization Modeling Objects
Home-page: http://pyomo.org
Author:
Author-email:
License: BSD
Location: c:\users\lbian\miniconda3\envs\dispatches-verify-1.3.0rc0\lib\site-packages
Requires: ply
Required-by: coramin, dispatches, gridx-egret, gridx-prescient, idaes-pse, omlt, rbfopt
---
Name: idaes-pse
Version: 2.0.0
Summary: IDAES Process Systems Engineering Framework
Home-page: https://idaes.org
Author:
Author-email:
License: BSD
Location: c:\users\lbian\miniconda3\envs\dispatches-verify-1.3.0rc0\lib\site-packages
Requires: backports.shutil-get-terminal-size, bunch, click, colorama, distro, flask, flask-cors, ipython, jupyter, lxml, matplotlib, nbconvert, nbformat, networkx, numpy, omlt, openpyxl, pandas, pint, psutil, pyomo, pytest, python-slugify, pyyaml, rbfopt, requests, scipy, sympy, tinydb, xlrd
Required-by: dispatches
---
Name: dispatches
Version: 1.3.0rc0
Summary: GMLC DISPATCHES software tools
Home-page: https://github.com/gmlc-dispatches/dispatches
Author: DISPATCHES team
Author-email:
License:
Location: c:\users\lbian\proj\dispatches-verify\dispatches
Editable project location: c:\users\lbian\proj\dispatches-verify\dispatches
Requires: dispatches-data-packages, dispatches-rts-gmlc-data, graphviz, gridx-prescient, idaes-pse, jupyter, nrel-pysam, pyomo, pytest, utm
Required-by:

adowling2 commented 1 year ago

@dguittet @radhakrishnatg @esrawli @nareshsusarla @Xinhe-Chen To recap, our goal is to have the scripts/notebooks used to generate key results from the reports included in the final release. If the scripts/notebooks are not in the release candidate, please work with @lbianchi-lbl to open and merge a PR.

adowling2 commented 1 year ago

@lbianchi-lbl I am following these instructions on an M1 macOS (with a recent branch on @nareshsusarla's fork) and got this error:

INFO: pip is looking at multiple versions of dispatches[surrogates] to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement tensorflow==2.10.0 (from dispatches[surrogates]) (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0)
ERROR: No matching distribution found for tensorflow==2.10.0

adowling2 commented 1 year ago

I decided to do pip install --editable . because I want to see if some scripts in @nareshsusarla's fork run on my computer (and tensorflow is not needed), so this is unimportant.

lbianchi-lbl commented 1 year ago

@lbianchi-lbl I am following these instructions on an M1 macOS (with a recent branch on @nareshsusarla's fork) and got this error:

INFO: pip is looking at multiple versions of dispatches[surrogates] to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement tensorflow==2.10.0 (from dispatches[surrogates]) (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0)
ERROR: No matching distribution found for tensorflow==2.10.0

Is this in a fresh Conda environment? If so, the Python version could be an issue - for various reasons DISPATCHES is only compatible with 3.8 or 3.9, with the latter being the one used for the instructions above.

Running pip install --editable . is a good alternative to pip install -r requirements-dev.txt, modulo some possible test failures due to e.g. addheader, but if things work, then no need to change.

adowling2 commented 1 year ago

Yes, a new conda environment with Python 3.9. I wonder if that version of tensorflow is not available for macOS arm.

lbianchi-lbl commented 1 year ago

Yes, a new conda environment with Python 3.9. I wonder if that version of tensorflow is not available for macOS arm.

Oh, right, sorry, for some reason I skipped the part about the M1 Mac part in your message. Yes, I'd say it's likely, especially considering that 2.10 is relatively old. From a quick check, it looks like the arm64 build is only available (on PyPI) since Tensorflow 2.13: https://pypi.org/project/tensorflow/2.13.0/#files

gmlc-dispatches / dispatches