Rework examples/src structure

[jghouse88]

Propose changing existing src structure for the following reasons:

• @dangunter and I have been a bit confused with the current set up of the src folder especially when working on IDAES/examples-dev#108 as to what constitutes as "workshop" and what constitutes as "tutorial".
• It is difficult for a new user to identify where to start: Tutorials or workshops.
• What is the recommended progression for a new user when looking at the examples?

Proposed new structure:

src
  tutorials
     basics
         Intro to Python and Pyomo
     unit_models
          Flash Unit Model
          Other unit models
     Flowsheets
          HDA Flowsheet
          Allows adding power plant and other flowsheets in future
     Advanced
          Property packages
          Custom model development
          Parameter estimation
              parameter estimation using NRTL 
              data_recon_and_parameter_estimation
     dmf
          dmf related notebooks
     matopt 
     surrogate

For clarity, the current structure is as follows:

src
  matopt
  surrogate
  tutorials
     unit_models
     advanced
        custom model development
        data_recon_and_parameter_estimation
     property packages
     tools
  workshops
       Intro to Pyomo
       Flash Unit Model
       HDA Flowsheet
       Parameter Estimation

Currently, the major distinction between workshop and tutorial notebooks is the existence of "exercise cells". With the new structure, we will end up with notebooks in a folder that will have an exercise version and a tutorial version. Might be confusing to the user (maybe) as to why some notebooks are like exercises and some are not.

Thoughts?

[andrewlee94]

First, I think this is long overdue and we do need to have a structure that helps users through the progression (plus a landing page which explains it in more detail). Notably, what we currently call workshops are really tutorials and what we call tutorials are more like examples.

Regarding the proposed structure, I would be inclined to keep the current "workshops" (0, 1 & 2 at least) in the basics section - I see these as the basic instruction on how to get started after which you can branch out. I also note there is a big difference in style and content between Module 1 and the unit model examples. To the point, Module 1 is clearly a teaching module, whist the others are simpler examples of a unit in use (and we don't have the resources to expand them).

So, my suggestion would be to clearly identify teaching units (workshops or actual tutorials) from simpler examples. They can possibly be grouped in to categories as @jghouse88 suggested, but I think we need to make the content clearer so people can easily find the tutorials before diving into examples. I.e. each set of examples should have a related tutorial which provides the necessary introduction (which could simply be the same content with deeper explanation).

[TimBartholomew]

I think the proposed structure is significantly better than what we have. There really was no distinction between tutorials and workshops, so combing that to tutorials makes a lot of sense.

I agree with @andrewlee94 about making a distinction between tutorials and examples. It's also what I envisioned when I did the reorg on idaes-pse readthedocs, where the link to the examples site is under "Tutorials and Examples".

I think the best way to make this distinction is just at the top level. Like the following:

src
  tutorials
    ...
  examples
    ...

I'd also probably squash the previous workshops in basic tutorials because creating a flowsheet and doing some simulation and optimization is the minimum of what a user would do in IDAES. Something like this possibly:

src
  tutorials
    basics
       intro to python and pyomo
       simulation - flash unit
       optimization - HDA flowsheet
    advanced
       custom property packages
          simple, detailed, ...
       custom unit models
          simple, detailed, 1D, ...
       parameter estimation
       dmf
         (dmf notebooks, if they are tutorials)
       modeling extensions
         (matopt, surrogate, if they are tutorials)
  examples
    unit models
      ...
    flowsheets
      ...
    dmf
      (dmf notebooks, if they are examples)
    modeling extensions
       (matopt, surrogate, if they are examples)

[jghouse88]

@andrewlee94 and @TimBartholomew I like the examples/tutorials distinction. I also thought of including the current modules 0, 1, and 2 under basics but then thought they are better grouped under unit models/flowsheets. However, I am fine with having it under basics.

Bottom line:

tutorials are learning notebooks with or without exercises with detailed explanation examples are just simple notebooks without exercises with less detailed explanation

I am fine with this distinction. If all agree we can do the reorg once the Sept release has been cut.

[andrewlee94]

So, an alternative would be to do as @jghouse88 suggested, but clearly indicate tutorial versus example. A crude way would be to preface tutorials with "Txx" and examples with "Xxx" - that way the tutorials will appear at the top of the directory tree for each section.

[dangunter]

Keep in mind that the installation of the examples with idaes get-examples already splits notebooks with 'exercise' cells into 2 notebooks, one with and without those cells. So tutorials and non-tutorials can be the same notebook.

I'm in favor of a flat structure, and we can guide users through it with the landing page. This gives us more flexibility without more reorgs in the future. I would suggest we drop not only the tutorials/examples structure, but also basic/advanced.

src
     Intro to Python and Pyomo
     unit_models
          Flash Unit Model
          Other unit models
     Flowsheets
          HDA Flowsheet
          Allows adding power plant and other flowsheets in future
      Property packages
      Custom model development
      Parameter estimation
              parameter estimation using NRTL 
              data_recon_and_parameter_estimation
     dmf
        ...
     matopt 
        ...
     surrogate
           ...

[andrewlee94]

@dangunter To give an example of the tutorial and non-tutorial thing, we have Workshop Module 1 which is a detailed tutorial of a Flash unit model, but we also have the far simpler examples that Anuja and @jghouse88 have done for other unit models in the library. So, it is not that they are the same basic notebook with different cells, but different levels of detail when developing the notebook.

So far the unit model example, a user should really go to the Flash unit tutorial first as it has a lot more detail and instruction, and can then go look at the simpler examples for units they might be interested it. These could quite easily live in the same folder with each other (they all relate to unit models), but there needs to be some indication that you should start with the more detailed tutorial first.

[dangunter]

I am working on something here.. I think we can squeeze it in before the code freeze? Happy to pick up on what others have said re: org. @jghouse88 @andrewlee94 @TimBartholomew please feel free to weigh in

[jghouse88]

This issue needs to be transferred to the open repo. @dangunter Shall I open this there?

[dangunter]

Implemented in PR #4