ODoE Tutorial Issue

[jmorgan29]

I am running the tutorial "Example 1: ODoE with Existing Candidate Set" in FOQUS version 3.16.0. I am able to complete the tutorial up to Step 11, when clicking on the "Run ODoE" button results in a TypeError associated with the file \foqus_lib\gui\sdoe\sdoeSetupFrame.py

For some additional background, I am following the tutorial instructions of using G-opt as the optimality criterion with a design size of 2 and number of restarts chosen as 3. There is also an additional method option for 'Fisher' and 'Bayesian', which seems to have been added in a later version than that used for the tutorial documentation. I have tested both methods and got the same error.

[lbianchi-lbl]

@jmorgan29 thanks for opening the issue. From a quick look, I think the problem could be due to how the PSUADE output is parsed in foqus_lib.framework.sdoe.odoeu.odoeu():

https://github.com/CCSI-Toolset/FOQUS/blob/5588689cdd64bc3d7270d129bb7c3330f4a6e0a1/foqus_lib/framework/sdoe/odoeu.py#L229-L241

More specifically, if the regular expression at L232 doesn't match, best_optval is left to its default value of None which causes the error in sdoeSetupFrame.runOdoe() since the format string expects best_optval to be a number instead.

Maybe @sotorrio1 can chime with more details and/or provide some example PSUADE output files/strings that we could use to add tests to exercise the parsing functionality.

[sotorrio1]

@jmorgan29 thanks for opening the issue, I'll definitely update the documentation to show the new method ("Fisher" vs "Bayesian") and the new optimality criterion E-Opt.

I just ran tutorial one for ODoE and I can run it without being able to reproduce your error. What version of PSUADE are you using? You should make sure you installed version 1.9, since 1.9 will be the only version supporting the newest version of ODoE. I will add a note to the tutorials as well to avoid confusions.

@lbianchi-lbl I'd love to help regarding adding some testing to the parsing functionality. We're not writing to disk but extracting the information from the PSUADE console, but I can definitely provide inputs.

[sotorrio1]

There is already a note regarding PSUADE 1.9:

https://foqus.readthedocs.io/en/stable/chapt_odoe/tutorials.html

[jmorgan29]

Thank you @sotorrio1 for investigating this. I checked the version of PSUADE that I am trying to use and have verified that it is psuade-lite 1.9.0

[sotorrio1]

@jmorgan29 I've also tried running ODoE on a Windows machine and I was successful. Just make sure you follow all the steps in the tutorial, specially clicking Done after step #6 of the tutorial.

[lbianchi-lbl]

@jmorgan29 I've also tried running ODoE on a Windows machine and I was successful. Just make sure you follow all the steps in the tutorial, specially clicking Done after step #6 of the tutorial.

Just as a side note, @sotorrio1 mentioned that is tangentially related to the windows stacking order issue on macOS since the ordering issue makes it harder to click Done. However, this doesn't apply to Windows (the OS) where the sample generation menu should remain where it's supposed to be.

[lbianchi-lbl]

Reopening this issue since we still need to figure out the original issue with the PSUADE output parsing.

[jmorgan29]

I checked again, verified that I clicked Done after step #6, and am still getting the error. I will check to see if I can find an additional tester for this tutorial to see if we can further diagnose the issue.

[sotorrio1]

@jmorgan29 could you share your psuadelog file that gets created in your FOQUS working directory?

[jmorgan29]

@sotorrio1, I checked the psuadelog file and this does include a couple of errors, one related to the candidate design set size and the other due to being unable to read the file for the evaluation set. psuadelog_jcm.txt

[sotorrio1]

@jmorgan29 thanks for sharing the file. It seems there is a mismatch with the size of the candidate set. The line in the psuadelog file that says Enter the file name of your candidate design set : Size of the candidate set = 373 should actually be 35 instead of 373. Make sure your candidate file is "CandidateSet.csv" and your evaluation file is "EvaluationSet.csv".

[jmorgan29]

Yes, this is the case. I also verified that the candidate set has a size of 35 and the evaluation set has a size of 373.

[ksbeattie]

@jmorgan29 have you been able to reproduce this?

[jmorgan29]

@ksbeattie, I am still getting the same issue when running this example, even after switching to FOQUS 3.17.1

[sotorrio1]

@jmorgan29 nothing has changed, I believe he was wondering if do you know other person or machine able to reproduce the same error. I still can't reproduce it. I can try to write the script to file and compare scripts... it might have something to do with file permissions though

[lbianchi-lbl]

One additional source of information that could help would be running FOQUS in an empty working directory and then attach its contents (as a .zip archive) to this issue when the error occurs.

[ksbeattie]

@jmorgan29 could you try again with the suggestion from @lbianchi-lbl above? Possibly working together with @sotorrio1 on it, real-time?

[jmorgan29]

@ksbeattie, @sotorrio1, @lbianchi-lbl, my apologies for the delay on responding to this. I created a new conda environment and installed the latest version of ccsi-foqus (3.18.0) using pip. I also created a new working directory for FOQUS. When running the tutorial, I am no longer getting this problem and I can now complete the tutorial.