1.7.4 MM Module Update - Githubissues

jamesmrollins commented 5 years ago

@lzim @TomRust @staceypark @saveth @branscombj @dlkibbe @dlounsbu

Here are the MM submissions.

All subscripts removed (all affected variables now have either “AUD ” “DEP ” “OUD ” or “OtherNeeds ” prefixes)
All user input variables changed to “Gaming”
All data connections updated
“data switch” updated to match functionality of CC
Model starts in equilibrium in week -13
Added Engagement Duration as New Experiment

Here is how it will look in the SIM UI

And a close up of the switch with alternative wording (same as recommended in CC):

And a close up of the Sensitivity of Referrals to Wait Time sliders:

Please everyone verify so I can send to Dev.

Thanks, James

branscombj commented 5 years ago

@jamesmrollins @TomRust @lzim

We don't have consistency: "start rate" vs "starting rate". Looks like we use the shorter word where the variable names get long (e.g. SP).
Is the range for the Engagement Duration switch (104) long enough? Is the minimum for AUD supposed to be 1 instead of 0?
I'm looking at the AGG model in the sim in Test version. I see the variable "New Patient Start Rate" in the Team Data Table but don't see a variable labeled that in the model diagram.
I'd like to suggest edits to i definition for New Patient Start Rate (AGG) - if I'm correct (please verify) that it means starting any service, brand new to the team?

lzim commented 5 years ago

MY RESPONSE FOR MM IS THE SAME AS CC

Hello @jamesmrollins @TomRust @staceypark @dlkibbe @branscombj @dlounsbu

NOTE: In PROD the MM module still says "Medication Management Model" at the top of the main section with the model diagram. Those blue bar headers should all just indicate the module name without "model" e.g., "Medication Management"

We do need this final and to DEV ASAP. Let's make sure we are ready for that by the end of the MTL Launch call today.

I believe the new switch could simply be this, no need to re-reference "team data" when that is already shown as an exogenous "Team Data" input below, and the slider is Red, which learners will know is from team data by convention. I believe this also means we will not need to include the word "use."

I am not convinced that calling this switch "Prioritize Patients" will be well-received by providers. We will further explain the operational definition (as above) in the "i" information text, what about just using "Prioritize" alone as the header, then showing where the data comes from as below:

"Prioritize" left = "New Patient Starting Rate" right = "Existing Patient Return Visit Interval"

In response to Jane's points:

We should be consistent re: start rate vs. starting rate - I think "New Patient Start Rate" is clear for the slider, then the model uses parallel black label rates on the flows: starting, completing and ending, etc. Thoughts?
No, 104 weeks is not a long enough duration, is there a reason it was set to 104? I would propose possibly extending the range to 208 (4 years).
Yes, let's make sure these red variables are in the team data table.
No, @branscombj, in AGG it does not necessary mean new to the team, but it does mean "new patient start rate" for a new service within a team. This is a key idea about the differentiation between the intake evaluate rate and review rate in the AGG model. Right @TomRust?

@jamesmrollins and @TomRust why a green slider named "Engagement Duration?" Using our style conventions, this should be a red slider named "Median Engagement" corresponding to the team data table. Right?

Thanks all!

Lindsey

jamesmrollins commented 5 years ago

Below are requested changes for discussion:

MM "Model" remove from section header.

Change color, title and range of values of "Median Engagement" slider.

Change title of switch in SIM UI to "Prioritize."

Change title and labels of switch on Experiments Section, and add "BC" button (map BC to Vensim variable "fRVI"):

jamesmrollins commented 5 years ago

TomRust commented 5 years ago

@branscombj -- Good questions! Here is my best shot at answers:

"Start rate" vs "starting rate". -- I don't know which is more important, parallelism vs brevity. No opinion.
Range for the Engagement Duration -- I don't know about the max for experimentation, but the minimum should definitely be 1 week, not zero.
AGG "New Patient Start Rate" Data Table -- "New Patient Start Rate" shouldn't appear in the model diagram, as we don't directly use that data in the AGG model (which is the total new patients scheduling rate across all services). We break that rate up into two groups -- patients new to the team who complete an official (ie, CPT-coded) intake visit (called Intake Rate), and those who start a service without an official intake visit (which we call the Review Rate). Our base case value for Intake Rate is based on the supply allocated to official Intake visit slots, and our base case value for Review Rate is calculated by subtracting the data on the new patient start rate from the supply (we assume that this remainder is made up of all the people who start a new service w/o an official intake visit). But, we should definitely show BC values for both rates (we have it for Intake, but not Review).
itext definition for New Patient Start Rate (AGG) - it means new to any service, not necessarily new to the team.

So: @jamesmrollins -- can we make these two changes: 1) set the minimum for all Engagement Duration sliders to 1 week 2) add a BC button to Review Rate slider (Vensim variable: "Data on Review Rate"

jamesmrollins commented 5 years ago

Below are a summary of requested changes in MM:

Change title bar to remove word "Model"

Change team data switch into "Prioritize" switch as shown. Change "Median Engagement in MM" to "Engagement Duration (median)(weeks)" in the Team Data Table. Base case variable will be mapped to “AUD fRVI," “DEP fRVI,” “OUD fRVI,” and “OtherNeeds fRVI” Vensim variables respectively.

Provide following "i" text for the switch:

“Control what data are used in the simulation - Select 0 for "RVI" to prioritize existing patients, where the team first sees all existing patients at the RVI estimated from the data, and only starts as many new patients as there are remaining open slots (based on RVI, supply, and missed appointment data). Select 1 for "New Patient Starts" to prioritize new patients, where the team first sees all new patients at the rate estimated from the data, and only sees existing patients at the RVI possible given the remaining open slots (based on start rate, supply, and missed appointment data).”

Add "Engagement Duration" slider in Experiments Section. Range minimum is 1 week to 4 years (208 wks). Zero "0" will signal base case values.

Provide following "i" text for the slider:

“Manually adjust the median number of weeks that patients receive medication management visits with the team. (wks)”
Modify the "Sensitivity of Referrals to Wait Time" slider to incorporate other variables in the "array":

Modify the Sim UI switch icon as shown below.

Add "Engagement Duration" to the Expanded Ouputs Section and to the Save drop down menu.
Delete "SAXW - Slots Allocation (With X Waiver), - Abbreviation truncates the same as ^Slots Allcation (No X Waiver)" highlighted below and add the following, "SAWXW - Slots Allocation (With X Waiver)" and add abbreviation "SANXW - Slots Allocation (No X Waiver)"

Modify "Sensitivity of Referrals to Wait Time" to the stacked icon style, to reflect the 4 services.

staceypark commented 5 years ago

@jamesmrollins I believe we previously punted this to the MM Sprint. Can you update the text in the Help Page Abbreviations to match the change made in the Save section for the abbreviations?

The below should now say "SANXW - Slots Allocation (No X Waiver)"

"SAXW – Slots Allocation (With X Waiver) ---- Abbreviation truncates same as ^ Slots Allocation (No X Waiver)"

jamesmrollins commented 5 years ago

Version 1.7.4 MM is in TEST: Please consult the design summary and review checklist below to and indicate your "Go, No-Go" response. I have truncated the checklist to only cover things that are applicable for this sprint. **

Design Summary: MM user-defined experiment variables were changed from variable arrays to individual gaming variables and can now time step. In the Experiments Section, the Team Data Table was edited to change "Median Engagement Duration" to "Engagement Duration (median)(wks)." The "Use Team Data" switch was replaced with a "Priortize" switch to enable the user to choose between using the "New Patient Start Rate" (default) or the "Existing Patient RVI" as the starting point. Under the Engagement Pattern Section, an "Engagement Duration" slider was added with a range of 1 to 208 wks, with the base case set as default. In the Main SIM UI, all instances of "Starting Rate" have been changed to "Start Rate." The "Median Engagement" red text has been replaced with an "Engagement Duration" slider icon. The "Use Team Data" switch was edited to say, "Prioritize." The SIM UI Section header was edited to remove the word "Model" from "Medication Management."

Simulation Run (James and Tom)

[x] 1. Widgets return same values as the client version of Vensim.
[x] 2. Charts are consistent with values from the client version of Vensim.
[x] 3. (If new experiments are added) New experiments function properly.
[x] 4. (If new functions are added) New functions operate as specified.

Outputs (if variables or charts are added) (James, Stacey and Savet)

[x] 1. The save-experiment naming function properly encodes any new variable in a manner that is understandable.

Training and Documentation (Debbie, Jane, and Stacey)

[ ] 1. Changes in simulation are noted in training documents, or are in process.
[ ] 2. Changes are consistent with dialogue and instruction in current video training.
[ ] 3. Help descriptions are up-to-date and include explanations of new functions or variables.
[ ] 4. "i" Information is up-to-date, correct and complete.

lzim commented 5 years ago

Thanks @jamesmrollins

@staceypark will be reviewing the training and documentation component. I will review MM in Test.

Thanks!

saveth commented 5 years ago

@jamesmrollins the test simulation Team Data Table for MM is not using the right standard unit for Engagement Duration.

The master crosswalk is correct, just not the sim. The same goes for CC Team Data Table. Otherwise, I'm done reviewing the master crosswalk for mm. 👍

jamesmrollins commented 5 years ago

@saveth do you mean that it should be "wks" instead of "weeks?"

saveth commented 5 years ago

@jamesmrollins yep. :)

branscombj commented 5 years ago

@jamesmrollins @lzim @TomRust @staceypark @dlkibbe I have not processed all of the back and forth above, but as I use MM in Test today, the old "use team data for start rate" switch which is now "prioritize" does not match the i information, which talks about zero and one settings:

lzim commented 5 years ago

@branscombj We do want to keep the 0 or 1 included in the "i" information, because when you see the experimental values you can interpret your selection, e.g., 1 = prioritize new patient starts, 0 = prioritize existing patient RVI and then use it in the save dialogue : mm_prioritize

However, @TomRust: a more "hard coded" question is: Would you say that the MM Module function "prioritize" is closer to a "mode" the way that "Team Data" and "Balanced" are in AGG?

new_patient_starts

Meaning the default that you choose for your bc needs to be made very explicit as the logic of this prioritize switch impacts the overall behavior of the model (multiple variables)?

I am proposing to @jamesmrollins that we may need a "Mode" line of the save dialogue between "Experiment Name" and "Variable" drop downs, to denote this type of overall decision that you may make even in the base case run, that would influence all runs.

save_dialogue

For any selection that impacts the overall logic/behavior of the model, it's important that the save dialogue force a run name pre-fix that clarifies "type" of "bc" (or other run) that it is, this is how I tried to save two base cases with the prioritize "variable," but I think a mode drop-down is clearer:

TomRust commented 5 years ago

@branscombj -- Good point about the itext -- it is both correct, and confusing! :)

How about these tweaks to the itext: Control what data are used in the simulation - Select "Existing Patient Return Visit Interval" to prioritize existing patients, where the team first sees all existing patients at the RVI estimated from the data, and only starts as many new patients as there are remaining open slots (based on RVI, supply, and missed appointment data) -- shown as 0 in Experiment Values. Select "New Patient Starts" to prioritize new patients, where the team first sees all new patients at the rate estimated from the data, and only sees existing patients at the RVI possible given the remaining open slots (based on start rate, supply, and missed appointment data) -- shown as 1 in Experiment Values.

@lzim -- I'm confused why you get different behavior with the switch positions. Both options should yield a model in equilibrium (just with different, but steady, RVIs and New Patient Start Rates). I just ran MM in forio, and it's definitely not producing the behavior in Vensim. I'll get with @jamesmrollins today to figure this out. :(

TomRust commented 5 years ago

@jamesmrollins -- also:

Where are the "xxx Actual RVI" variables in the Results Dashboard dropdown list?
Where is the arrow on the model diagram that points at Completion Rate?

I've also got a 5 "sum" variables in the model that I thought we were adding to the drop down list, so users could see across the entire team, not just by patient cohort:

Total Slots Allocation % (without X waiver)
Total Slots Allocation % (with X waiver)
Total Current Supply
Total RVI
Average Actual RVI

branscombj commented 5 years ago

@TomRust That works for me. @dlkibbe @staceypark @lzim Does this sound clear to you?

How about these tweaks to the itext: Control what data are used in the simulation - Select "Existing Patient Return Visit Interval" to prioritize existing patients, where the team first sees all existing patients at the RVI estimated from the data, and only starts as many new patients as there are remaining open slots (based on RVI, supply, and missed appointment data) -- shown as 0 in Experiment Values. Select "New Patient Starts" to prioritize new patients, where the team first sees all new patients at the rate estimated from the data, and only sees existing patients at the RVI possible given the remaining open slots (based on start rate, supply, and missed appointment data) -- shown as 1 in Experiment Values.

TomRust commented 5 years ago

@jamesmrollins -- Found one more! Drop down list is also missing for each cohort:

Measured versus Actual Time

TomRust commented 5 years ago

@jamesmrollins @lzim -- I think the switch is being activated at time zero in Forio, instead of at the beginning of the run (as it is in Vensim). Activating at time zero just changes the RVI to match the estimate from the data, without starting the model with the correct number of patients in each stock for that RVI. This is not a realistic scenario at all.

@jamesmrollins -- Can we get the switch to work like it does in Vensim, where the value is set before the sim starts running? (for MM and CC...if anyone cares about CC)

TomRust commented 5 years ago

@jamesmrollins -- one picayune detail -- the units are wrong for "New Patient Wait Time" It should be in weeks not patients.

jamesmrollins commented 5 years ago

@TomRust - I just went through my documentation. Somehow those changes got dropped off when I prepared instructions for DEV. In other words, I screwed that up when we went from a single 1.7 to 1.8 Development thread to break outs by model. These will be corrected tonight. I will also conduct a complete audit of all threads and documents to ensure I have captured everything.

Let me summarize what we need below:

"i" changes for "Prioritize Data" switch:

"Control what data are used in the simulation - Select "Existing Patient Return Visit Interval" to prioritize existing patients, where the team first sees all existing patients at the RVI estimated from the data, and only starts as many new patients as there are remaining open slots (based on RVI, supply, and missed appointment data) -- shown as 0 in Experiment Values. Select "New Patient Starts" to prioritize new patients, where the team first sees all new patients at the rate estimated from the data, and only sees existing patients at the RVI possible given the remaining open slots (based on start rate, supply, and missed appointment data) -- shown as 1 in Experiment Values"

Add the following variables to the Results Dashboard drop down list:

xxx Actual RVI
Total Slots Allocation % (without X waiver)
Total Slots Allocation % (with X waiver)
Total Current Supply
Total RVI
Average Actual RV
xxx Measured versus Actual Time

Add arrow on the model diagram that points at Completion Rate? I assume it is VA Light Blue.
Double check units in the charts - (exg., New Patient Wait Time).

Results of Conversation with @TomRust regarding model equilibrium differences between modes in MM.

@TomRust agrees with @lzim that a "Mode" drop down in the Save menu for CC and MM would be clearer. Choices would be "New Pt Start Rate" and "Existing Pt RVI" for MM and "Use Team Data" for CC.
The model cannot rerun a different equilibrium without resetting all the Experimental values. Therefore, we may have to put the mode switch in a dialogue when the user selects MM or CC from the Home page and selects Play. That way we can gather the mode information prior to the model running to equilibrium. @jamesmrollins will investigate with DEV tonight.

staceypark commented 5 years ago

@jamesmrollins MM Prioritize Switch Changes

Rename "Prioritize" to Priority Mode
"New Patient Start Rate" will now be on the right hand side of Priority Mode graphic, and remain the value of 1 for model logic. Add the number "1" under the slider bar
"Existing Patient Return Visit Interval" will now be on the left hand side of Priority mode graphic, and remain the value of 0 for model logic. Add the number "0" under the slider bar
Save dialogue: Remove "Prioritize" from Variable dropdown. Keep recent addition of Mode dropdown

jamesmrollins commented 5 years ago

@staceypark @lzim @TomRust

Current Situation: The MM Priority Mode switch is currently not operating as per @TomRust design intent. The problem is that each mode has a different equilibrium, and to achieve equilibrium we have to run from a negative time value to zero. If we switch the mode from default to Balanced at time zero, we would have to back up, which is impossible. In order to back up, we have to reset the model. The model is presented at time zero and at equilibrium of the default priority mode setting.

Possible solution?: DEV and I are trying to determine if there is a workaround. One possible alternative is to manually run the model through equilibrium. This may enable the user to set the appropriate mode before equilibrium.
Implications:

I believe the Experiment Timeline functions are global. This may cause us to make the equilibrium function manual for all models.
We have the same equilibrium problem also for CC. Do we also want to change the switch as described in your earlier post?

Once I have a workaround, and your approval, I will make the changes you have stipulated above.

jamesmrollins commented 5 years ago

Here is a mock up of the switch you described above. Unless I hear otherwise, I will include in tonight's pull.

staceypark commented 5 years ago

@jamesmrollins seems right to me!

jamesmrollins commented 5 years ago

@lzim @staceypark @TomRust Although we closed the CC thread, shouldn't the logic above also pertain to the CC model? @TomRust and I discussed how the equilibrium is also affected by the default in the CC model. Wouldn't the "i" information also be the same? I will assume this to be the case and make these changes unless directed otherwise. ~James

lzim commented 5 years ago

@jamesmrollins

Great catch!

Yes, it should. Re-opening the CC issue now.

Any update on aligning Forio and Vensim differences in when/how the priority mode switch “kicks in” from initialization to today (i.e., negative time to equilibrium)?

As soon as we have a fix, we will address (as necessary) for the learner in associated documentation (guides/.gifs, cheatsheets, help, etc). Especially if it means learners will have to make this decision for MM and CC modules in the home screen dialogues before model initializes. Although, fingers crossed there may be a better fix than that!

FYI @TomRust and @staceypark

TomRust commented 5 years ago

Hi @jamesmrollins -

I like what you've currently got in DEV for the "initialize the model with what data" issue.

How about this for additional text explaining what we are asking them to choose?

lzim commented 5 years ago

@TomRust and @jamesmrollins

If this is the route that has to be pursued for the MM issue, then it needs to clarify in the pop-up that it is only for MM module.

Needs to say "Priority Mode" to reflect the label on the switch.

Recommended edits:

"Selecting the Existing Patient Return-to-clinic Visit Interval (RVI) for Priority mode, prioritizes seeing existing patients at the RVI estimated from team data, and will only start new patients in remaining open slots after existing patient RVI, appointment supply, and missed appointments are taken into account."
"Selecting the New Patient Start Rate for Priority Mode, prioritizes seeing all new patient at the rate estimated from team data, and will only see existing patients in the remaining open slots after the new patient start rate, appointment supply, and missed appointments are taken into account."

Tom can you help with this:

Needs to include details about what the implications of each choice for experiments.
Need to include some instruction about the conditions under which the learner would select each.

Thanks!

jamesmrollins commented 5 years ago

@lzim @TomRust Please take a look at the concept below. I would like to pull this into DEV tonight. @lzim, I believe this window will only pop up for MM and CC models that require the pre-start Priority Mode selection. So it may not be necessary to indicate that it is MM or CC, since they have to choose that before this pop up is presented. I'm going to take a swing at the Team Data Table next.

jamesmrollins commented 5 years ago

@lzim @TomRust Here is the Team Data Table with Priority Mode Values.

lzim commented 5 years ago

@jamesmrollins @TomRust

Thanks, I keep forgetting it is both CC and MM.

Priority Mode Pop-up:

With this bold formatting header, I think you can edit the capitalization to: "By selecting this priority mode, ..." No need to capitalize "selecting," "priority," or "mode."

Team Data Two minor edits from me:

It would be nice if the equal signs aligned vertically.
Missing a comma before "then" in: "If Existing Patient RVI was selected, then..."

jamesmrollins commented 5 years ago

Check!

OK - Off to DEV

branscombj commented 5 years ago

Apologies - I'm probably way too late with this suggestion:

Priority Selected	Base Case RVI
Existing patient RVI
New patient start rate

lzim commented 5 years ago

Thanks @branscombj

I believe that your recommendation would not reflecting the complexity of the Priority mode selection without an additional column like this:

Priority Selected	Base Case RVI (wks)	Base Case Start Rate (pts/wk)
Existing patient RVI	--	3.17 pts/wk
New patient start rate	15 wks	--

Right? @TomRust and @jamesmrollins

It is confusing to try to understand what would happen in two the four values for RVI and/or New patient start rate in the Team Data Table rows above the Priority Mode buttons...

We'll need some really clear "i" information.

branscombj commented 5 years ago

I believe that your recommendation would not reflecting the complexity of the Priority mode selection without an additional column like this:

Ah, I see. Need the opposing units.

Priority Selected	Base Case
Existing patient RVI	New patient start rate = 3.17 pts/wk
New patient start rate	RVI = 15 wks

jamesmrollins commented 5 years ago

@branscombj @lzim @TomRust So have all of you decided that you want that table added to the Priority Mode pop-up? If so, where will it go? Just below the buttons? Let me know.

Also, if you want to see the current scheme for CC and MM, it is up in Test. I recommend a review and then I can make adjustments tonight. If we keep the ball rolling, I may be able to get this all done by Thursday.

staceypark commented 5 years ago

@jamesmrollins -- also:

Where are the "xxx Actual RVI" variables in the Results Dashboard dropdown list?

Where is the arrow on the model diagram that points at Completion Rate?

I've also got a 5 "sum" variables in the model that I thought we were adding to the drop down list, so users could see across the entire team, not just by patient cohort:

Total Slots Allocation % (without X waiver)

Total Slots Allocation % (with X waiver)

Total Current Supply

Total RVI

Average Actual RVI

@lzim @TomRust @jamesmrollins it looks like these aren't alphabetized in the dropdown list. They're alphabetized, but after the patient cohorts items. Is this what we decided on as the easier to find route?

staceypark commented 5 years ago

We'll need some really clear "i" information. @lzim @TomRust @jamesmrollins @branscombj where is this "i" information going? as part of the Team Data "i" or will Priority Mode Base Case Values get it's one "i" button

staceypark commented 5 years ago

@jamesmrollins the Priority Mode pop up successfully only comes up for CC and MM, but it does not come up on first instance - i.e. when there's no current session. It only comes up after when you're changing models.

TomRust commented 5 years ago

@jamesmrollins @lzim @branscombj --

The only place for itext about this is now the Team Data Table. How about:

Base case values estimated from the team's data, to correspond with the data selection made before opening the model. Each value ensures that the team will be able to maintain the historical estimate selected by the user, given all other team data (i.e, estimates of appointment supply, engagement duration, and missed appointment %).

Can we get a clearer name for the pop-up than "Priority Mode"? The user is selecting which team data to experiment from, based on their perception of the validity of the historical parameter estimates. Can we go with "Data Selection" or "Parameter Validation"? "Priority Mode" makes it sound like wwe're asking them to tell us which estimate is more important than the other...but the choice we are asking them to make is really black and white -- either they use RVI or they use Start Rate to initialize the model. The choice isn't about which parameter is more important, but which is more accurate -- which parameter should be used, and which discarded.

branscombj commented 5 years ago

Can we get a clearer name for the pop-up than "Priority Mode"? The user is selecting which team data to experiment from, based on their perception of the validity of the historical parameter estimates. Can we go with "Data Selection" or "Parameter Validation"? "Priority Mode" makes it sound like wwe're asking them to tell us which estimate is more important than the other...but the choice we are asking them to make is really black and white -- either they use RVI or they use Start Rate to initialize the model. The choice isn't about which parameter is more important, but which is more accurate -- which parameter should be used, and which discarded.

Or is it which one to hold constant?

jamesmrollins commented 5 years ago

@staceypark

See arrow below. Are you saying your browser did not display the arrow? It is activated on the Balancing Patients build.

Regarding the 5 Sum variables. Yes, it was decided to not mix them in with the patient cohorts.
I am not clear what issue you are trying to communicate regarding the Priority Mode dialogue. A user is either joining a current session, which is at t = > 0, or is starting a new one. If joining a current session, the Priority Mode dialogue will not present, because the priority has already been selected by another user. Let me know if I got this wrong.

@lzim I see the following decisions needing to be made:

Title of the "Priority Mode" dialogue box.
"i" information text for inclusion in the Team Data Table "i" information pop up. Remember that Team Data table "i" information is not dynamic. It has to be coded in.
Whether or not to add the complexity table recommended by @branscombj , and if so, where in the pop-up do we put it?

lzim commented 5 years ago

@staceypark and I are just turning back to this MM Issue #309 now after a long day 😅 Post 1/n

Thanks @jamesmrollins @branscombj and @TomRust for keeping this going today in the interim.

1. Priority Mode or something else:

To us "modelers" or "systems SMEs", I think this statement from Tom is the clearest: "they use RVI or they use Start Rate to initialize the model." I think this statement from Jane is also very clear "choose which one to hold constant."

At the same time, most providers will not know what making the decision to initialize the model with one (or the other) means, or what making the decision to hold one (or the other) constant means; parameter validation is even worse 😝. The problem in this case is serious, because due to Forio's limitations, they will be faced with this right up front before they do anything else in MM or CC.

Providers are using simulation to learn how to make better team improvement decisions, not learn modeling. And, they aren't saying which is more important clinically with this choice. But the implications of this decision are for the entirety of their CC and MM session and will never be compared.

I believe that framing this as a "mode" decision made for learning purposes helps translate use of the mode for understanding the tradeoffs of clinical problems. It doesn't swing too far toward suggesting it has [unrealistic] clinical implications it doesn't have, nor focus on data/model details that may undermine relevance/ease of use. We can help MTL Live learners with the choic, but not MTL Demo learners.

Recommended edits: Title: Learning Mode Existing Patient Return-to-Clinic Visit Interval

Selecting this mode prioritizes seeing existing patients at the Return-to-Clinic Visit Interval (RVI) estimated from team data, and only starts new patients in remaining open slots after the existing patient RVI, appointment supply, and missed appointments are all taken into account.

New Patient Start Rate [NOTE: I removed the "for Priority Mode]

Selecting this mode prioritizes seeing all new patients at the start rate estimated from team data, and only sees existing patients in remaining open slots after the new patient start rate, appointment supply, and missed appointments are all taken into account.

@jamesmrollins Stacey's 3 above, is happening for me too in TEST.

On my first instance of selecting the MM mode in Chrome just now, I was NOT prompted with a "priority mode" dialogue.
When I go back to home, and select CC instead, I DO get prompted, however.

Posting for @jamesmrollins more to come

staceypark commented 5 years ago

@branscombj Does @jamesmrollins response above resolve the arrow issue?

Regarding the 5 Sum variables. Yes, it was decided to not mix them in with the patient cohorts. @jamesmrollins ok great that makes sense

lzim commented 5 years ago

@jamesmrollins @TomRust @branscombj

@staceypark and I are grappling with which display is clearest. But, do believe Team Data details are needed in the pop-up as we expect that most clinicians will want to easily see the relevant data when making this selection. And, I do not believe they should be introduced to "Base Case" with this selection, rather it should just clarify at this stage of their learning what the initial "Team Data" estimates will be.

CC

Mode Selected	Team Data for Existing Patient RVI (wks)	Team Data for New Patient Start Rate (pts/wk)
If Existing Patient Return-to-clinic Visit Interval (RVI) mode, then	--	3.17 pts/wk
If New Patient Start Rate mode, then	15 wks	--

MM

Mode Selected	AUD	DEP	OUD	Other
If Existing Patient Return-to-clinic Visit Interval (RVI) mode, then Team Data for New Patient Start Rate (pts/wk) =	22.48	124.18	9.13	549.66
If New Patient Start Rate mode, then Team Data for RVI (wks) =	2.13	6.38	1.32	21.67

lzim commented 5 years ago

@jamesmrollins I think the tables should go under the text description of each mode

lzim commented 5 years ago

@jamesmrollins I'm still working on this "i" information.

Since it is dynamic now that we've made it a "bc" pop-up with the data tables above, then we have a little more time on this edit. But, here is my recommended edit.

These base case values estimated from the team's data corresponds with the learning mode selected when opening the simulation.

Selecting the Existing Patient Return-to-clinic Visit Interval (RVI) mode prioritizes the existing patients Return-to-Clinic Visit Interval (RVI) estimated from team data, and only starts new patients in remaining open slots after the existing patient RVI, appointment supply, and missed appointments are all taken into account.
Selecting the New Patient Start Rate mode prioritizes seeing all new patients at the start rate estimated from team data, and only sees existing patients in remaining open slots after the new patient start rate, appointment supply, and missed appointments are all taken into account.

lzim / teampsd

1.7.4 MM Module Update #309