OUU: Value of objective function

[fsoepyan]

I did 3 example problems using the OUU tab in FOQUS, including the example problem for Issue #24, which uses a JSON file from SimSinter and a simulation from Aspen Custom Modeler (ACM).

For all 3 example problems:

• The result for the best value of the objective function, presented by the OUU tab, is about -0.85.
• When I used Flowsheet to run the simulation using the decision variables suggested by the OUU tab, the value I got for the objective function is not -0.85, but something different.

[sotorrio1]

Hi @fsoepyan

In your example #2 you said you declared X01-X04 Z1 and the rest of your variables Z4. If you want to use 'PosteriorSampleNew' you need 13 variables but if you use your full flowsheet you have more than that (X05-X14 and Y01-Y07). What am I missing?

[fsoepyan]

• The file 'PosteriorSampleNew' is for Example 1
• X01 to X04 are Z1 and the rest of the variables are Z4 are also for Example 1

For Example 2 (the one with BFB in the file name and node name):

• x01 to x14 are Z1 (decision variables)
• y01 to y07 are Z4 (continuous uncertain or random variables)
• I assumed that y01 and y07 have "Uniform" distributions
• Under "Optimization Setup": I selected "Mean of G with respect to Z3 and Z4" (the default option)
• Under "Outputs": I selected "BFB.Cost_coe_obj" as "Objective Function", and I set all the other variables as "Not used"
• Under "UQ Setup": I selected "Generate new sample for Z4", chose "Latin Hypercube" as "Sample Scheme", and set 20 as "Sample Size"
• I clicked "Run OUU"

[sotorrio1]

Got it. Thanks!

[ou3llnl]

@fsoepyan : What value do you get when you run in flowsheet? Is it very different? Also, some models have some inherent randomness. Is that true in this case?

[fsoepyan]

• I ran OUU using the example file from Issue #24
• The OUU tab says that the values of the decision variables d1 to d4 are: 1, 1, 1, and 0.5
• The OUU tab says that the value of the objective function is -0.85
• I ran Flowsheet using the above values of the decision variables, and the average values of the uncertain variables
• The value for the objective function is -0.94

[fsoepyan]

• I ran OUU using the example BFB file that I referenced in this issue
• The OUU tab says that the value of the objective function is -0.85
• I ran Flowsheet using the values of the decision variables from the "Best So Far" box in the OUU tab
• I used the default values of the uncertain variables
• The value for the objective function is 138.8

[ksbeattie]

@sotorrio1 is working with a new version of psuade, will keep us posted.

[fsoepyan]

• I ran OUU using psuade 1.7.12
• For my example problem (same example problem from Issue #24), according to the OUU tab:

Objective Value: -5556.319 d1 = 0.999657 d2 = 0.944902 d3 = 0.944902 d4 = 0.562676

• I ran FOQUS using Flowsheet and using the above values for d1 to d4 (the decision variables)
• I kept the uncertain variables to their deterministic values
• The value I got for z (the objective function) is: -0.829835806133759 (different from what the OUU tab says)

[fsoepyan]

• I used the OUU tab using psuade 1.7.12 for the Example 1 file that I used earlier for this issue
• According to the OUU tab:

Objective value = -5556.319 x1 = 3843.996 x2 = 1124.004 x3 = 0.163578 x4 = 0.2516

• I ran Flowsheet using the above values of x1 to x4, and the default values of the uncertain variables
• The value I got for the objective function is: -102.67687903045

[ngbrenda]

OUU returns x and f(x). When the user enters x* into the flowsheet (which evaluates f), a different answer was returned. Need to check this out.

[ksbeattie]

@sotorrio1 will follow up on this one

[fsoepyan]

• I tested the OUU tab using the same example Aspen Plus file from issues #126 and #146
• I have 3 decision variables, 2 uncertain (UQ) variables, and I ran OUU using 25 samples
• According to the OUU tab:

Iteration = 1700 Objective value = -1041.183 Decision variable 1 = 0.95 Decision variable 2 = 301.09 Decision variable 3 = 199.29

• I went to the Flowsheet, and ran FOQUS using the above values of the decision variables, and the deterministic values of the uncertain variables
• The results I got was: Objective value = -1041.245
• The numbers are very close to each other, with a bit of difference, which is expected
• I will test the OUU tab using another example file

[fsoepyan]

• I used another example file to test the OUU tab (I used the OUU toy problem from the FOQUS GitHub download: FOQUS-master\examples\OUU\Toy2)
• I ran OUU to minimize z, while setting d1-d4 as decision variables (Z1), and omega1-omega4 as uncertain variables (Z4)
• These are the results I got from OUU when I ran it with 20 samples:

Objective value = -5556.32 d1 = 0.9997 d2 = 0.9449 d3 = 0.9449 d4 = 0.5627

• When I ran the Flowsheet using the above values of d1-d4, and the deterministic values of omega1-omega4, I got:

Objective value = -0.8298

• I also ran OUU with 25 samples, and these are the results I got from the OUU tab:

Objective value = -1041.18 d1 = 1.0288 d2 = 1.4572 d3 = 0.9937 d4 = 1.4612

• When I ran the Flowsheet using the above values of d1-d4, and the deterministic values of omega1-omega4, I got:

Objective value = 11.32552

• Not only are the results from OUU and Flowsheet inconsistent, but the results from OUU with 20 samples and OUU with 25 samples are also inconsistent

[ksbeattie]

@sotorrio1 says he is blocked on this by #147

[sotorrio1]

@fsoepyan I can't exactly run the problems you're mentioning (because of issue #148) but I've done some testing with toy problem Rosenbrock No Vectors (foqus_lib/examples/UQ). If you run OUU using x1, x2, x3 as Z1 and x4, x5, x6 as Z4; loading a 20 samples file for Z4 making sure the value for each variable is the same across the 20 samples (something that doesn't happen when you ask psuade to randomly generate the samples for you). Then if you use those values for your input variables in the flowsheet, you will get the same objective value. My point is that running OUU vs running flowsheet doesn't mean you will get the same results since you're doing single runs in the flowsheet and using samples in OUU.

[fsoepyan]

• I ran OUU for an example Aspen Plus file
• I have 3 variables set as Z1, and 2 variables set as "Fixed"
• The solution generated by the OUU tab is:

Iteration: 60 Objective Value: -111111 conv: 0.6 H2_flow: 293.1874 TOL_flow: 294.6004

• I went back to "Flowsheet", and set the variables equal to the OUU solution
• When I ran "Flowsheet", I got an error message: "The simulation completed with an error 1, Failure in node calculation, check node status"
• However, when I ran "Flowsheet" using the initial values of the variables, the run was successful
• This implies that the OUU solution produces an error for the Aspen Plus file
• Then, I went to the "Flowsheet" table, and according to the "Flowsheet" table, the optimum solution occurs at:

Objective Value: -1724.61 conv: 0.95 H2_flow: 295.55 TOL_flow: 249.86

• This is different from what is shown in the table in the OUU tab

[sotorrio1]

@fsoepyan So this problem doesn't require turbine? I might be able to reproduce your error if you send me the necessary files to run it. Thanks!

[fsoepyan]

Actually, this problem requires turbine

[sotorrio1]

I can't mimic this problem, but for this issue I want to reference my last comment (Oct 15, 2018, 5:31 PM PDT) on issue #269

[ksbeattie]

@sotorrio1 is waiting for @fsoepyan to reproduce this.

[fsoepyan]

I will close this issue. Please see my comments from 11/7/18 on Issue #269.