lzim / teampsd

Team PSD is using GitHub, R and RMarkdown as part of our free and open science workflow.
GNU General Public License v3.0
9 stars 23 forks source link

Bug Sim UI Team Care: The saving dialogue is saving an RVI experiment for MM with "ED" which should be for engagement duration #2852

Closed lzim closed 1 year ago

lzim commented 1 year ago

FYI @jamesmrollins
Let's talk about this at support_workflow or sim_ui_workflow

In the MTL Blue team that I co-facilitate on Thurs (wrapping up 02/02) I see that an experiment adjusting the RVI from 9 weeks to 12 weeks is labeled with "ED."

Shouldn't that be for engagement duration?

image

Also at wk4 jan_epic @emilymetcalf114 @epearman @lzim

Noticed that the experiment save order does not reflect the order of the drop down menu of changes made in the save dialogue See example of intake below.

image

jamesmrollins commented 1 year ago

@lzim @epearman @emilymetcalf114 @hirenp-waferwire

Problem

When the user saves an experiment, there is an inconsistency between the saved filename string and the actual saved variable and value.

Hypothesis

If the user saves the experiment multiple times before the run, the algorithm may not be accurately recording the experiment settings. Thus, when the experiment is run and saved, the saved filename string is incorrect.

Experiment

  1. Open Medication Management Module. Change the setting of an experiment variable. Then save the experiment without running the experiment.

image

  1. Change the experiment setting of original variable and add in another experiment variable setting. Save the experiment without running the experiment.

image

  1. Change experiment settings from original setting on ED panel to base case. Save the experiment. The "edoud" and the "edo" value calculations show negative values. Since they are on "BC," they should not be reported in the save name string.

image

  1. Run the simulation. Change the RVI value. Click save. The Sim UI, now includes the post-run changes to the experiment variables in the save string. RVI change was not a part of the run.

image

Conclusions

  1. In the use case above, the user may have made a post run change and then clicked save. Since the Sim UI does not lock out the experiment sliders post-run, this is a failure mode.
  2. In the experimentation, the Sim UI does not appear to be gathering some experiment sliders setting properly (ED), indicating another potential failure mode.

Recommendations

  1. Lock the experiment sliders after the run, so they do not get saved improperly.
  2. Investigate the ED issue at the code level and determine cause.
  3. Double check all the other sliders in all the modules to ensure they are gathering data correctly for the experiment name string.

Needed Decision

In which order would we like the variables to appear? Experiment variable order or dropdown order? Both were specified during the design phase.

image

lzim commented 1 year ago

@jamesmrollins

Does that make sense to you?

jamesmrollins commented 1 year ago

@lzim

lol.

Looks like we have an even split (with "?" list consistent with the Data UI).

image

Deeper Dive

lzim commented 1 year ago

Add to wk1 feb_epic support_workflow agenda

@epearman @emilymetcalf114 @jamesmrollins @lzim

emilymetcalf114 commented 1 year ago

@lzim @jamesmrollins @epearman Let's move to wk1 feb_epic sim_ui_workflow agenda tomorrow instead, since Lindsey and I will miss support_workflow. Thanks!

ghost commented 1 year ago

@lzim @emilymetcalf114

Discussed during support_workflow wk1 feb_epic

ghost commented 1 year ago

Discussed at wk1 feb_epic sim_ui

DECISION Hold on this issue since we need to review and align it with UAD and bookability

crossref #2851 - ASAP Due Bug: Re-Review the Binning of Encounters for Time Assumptions

1. Discussion - Evaluate in relation to UAD - bookability and labor mapping - wk1_feb mixed_methods

lzim commented 1 year ago

@epearman @emilymetcalf114 @jamesmrollins

I'm surprised @jamesmrollins is the only one to review UAD bookability??