Pyomo.DoE Development Plan

[adowling2]

The purpose of this meta issue is to track our development plans for Pyomo.DoE.

Immediate (November/December 2022):

• [x] Merge version 1 of Pyomo.DoE, #2294
• [x] Merge Pyomo.DoE logo, #2527

Pyomo.DoE Version 2 (start review in March 2023):

• [x] Remove extra index (using blocks)
• [x] Deprecate/remove sIpopt (pending discussion)
• [x] Update to use the TicTocTimer, see https://github.com/Pyomo/pyomo/pull/2294#discussion_r1024669360
• [x] Replace string options with Enum, see https://github.com/Pyomo/pyomo/pull/2294#discussion_r1024665131
• [x] Documentation and tutorial enhancements
• [x] Allow users to specify the names of the index sets (e.g., do not require time to be hard-coded as t, see https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025732006, https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025464771)

Pyomo.DoE Version 3+:

• [ ] Generalize measurement variance to consider correlations (not just diagonal)
• [ ] Add grey box objective function calculation
• [ ] Support decomposition with parapint
• [ ] Refactor interface for Pyomo.DoE and parmest to not define extra information (e.g., parameters, measurements) using strings, see https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025732006

[adowling2]

Please assign this issue to @jialuw96 and @hglynch

[jialuw96]

• [x] user define model time name https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025464771
• [ ] Create simple names for variables; https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025732006
• [ ] Get away from the two main indexed / unindexed branchs; https://github.com/Pyomo/pyomo/pull/2294#discussion_r1025732006

[jialuw96]

Pyomo.DoE Version 2 (Some other problems for discussion and enhancements, each one should be a small PR):

• Which is a better way to grab pyomo model variable values?
  • Get rid of eval;
  • Use getattr (Need string name)
  • Use ComponentUID as in https://github.com/IDAES/idaes-pse/blob/274e58bef55f2f969f0df97cbb1fb7d99342388e/idaes/apps/uncertainty_propagation/sens.py#L296 (Need string name)
  • find_component (Need handle to Var)
  • How to flatten variables with multiple indexes, for e.g. m.C['CA', 'A1', 16, 2]?
    • I saw the use of UnindexedComponent_Set and add_component. Is this a good way?
  • Is there something like a textbook for good development of a Pyomo package that I can refer to? (Or maybe a well developed contributed package that I can use as reference) I have the Pyomo cookbook but it is for users I think.

[blnicho]

• [ ] Improve test coverage (currently at ~66% but would prefer 85-100% coverage)

[jialuw96]

Note with @blnicho from IDAES meeting

• Create block objects as way 3 in the figure
• In user create_model(): return not only m, but also Var data to parameters, design variables, and measurements as in https://github.com/Parapint/parapint/blob/fad622d912c03d7c23b70241d5d18b96942a4d4b/parapint/examples/burgers.py#L173
• In this way we can handle multiple indexed variables in a safe way

[adowling2]

Update documentation:

• [x] @adowling2 careful read through
• [x] Incorporate new Pyomo.DoE logo

[adowling2]

@blnicho @jsiirola Is Monday, Feb. 6 a reasonable cut-off date for minor PRs, e.g., updates to Pyomo.DoE documentation, before the next Pyomo release?

My thought is this gives the dev call on Feb. 7 to wrangle outstanding reviews and merge.

[blnicho]

@adowling2 yes Feb. 6 is fine for a cut-off date.

[adowling2]

@blnicho @jsiirola FYI, @jialuw96 and I are working on using blocks in Pyomo.DoE in https://github.com/jialuw96/pyomo/pull/15 which is a PR from one branch in @jialuw96's fork into a second branch on her fork. Once this is ready for review from the Pyomo team, we'll open PR into pyomo/main.

[jialuw96]

Pyomo.DoE version 3+:

• [ ] add the integrator on an already discretized model.
• [ ] reduce_collocation_points https://github.com/Pyomo/pyomo/pull/2794#discussion_r1184419278
• [ ] a better name for calculate_FIM, which I temporarily name as result_analysis now
• [ ] check time index https://github.com/Pyomo/pyomo/pull/2794#discussion_r1197108238
• [ ] figure_drawing is not intuitive. https://github.com/Pyomo/pyomo/pull/2794#discussion_r1197140478
• [ ] Example tests should have assertions https://github.com/Pyomo/pyomo/pull/2794#discussion_r1197153950
• [ ] use repr() instead of str() https://github.com/Pyomo/pyomo/pull/2794#discussion_r1197110598

[adowling2]

Pyomo.DoE version 3+:

• [ ] Add tests for scenario.py https://github.com/Pyomo/pyomo/pull/2794#discussion_r1200780723
• [ ] Add README file or top comment for each example https://github.com/Pyomo/pyomo/pull/2794#discussion_r1200769511
• [ ] Choose better function names to eliminate confusion between compare and compute https://github.com/Pyomo/pyomo/pull/2794#discussion_r1196803473

[djlaky]

@adowling2 @blnicho Any objections to closing this in favor of the new issue I made today (#3345)?