search

lzim / teampsd

Team PSD is using GitHub, R and RMarkdown as part of our free and open science workflow.
GNU General Public License v3.0
9 stars 23 forks source link

wk4 june_epic MTL Manual Edits - Suicide Prevention and Stepped Care floors and episode counts #648

Closed holbrooa closed 3 years ago

holbrooa commented 5 years ago

@TomRust and @holbrooa are meeting to discuss emergent SP parameter issues.

  1. High-symptom proportions are limited to user-selected SMH team Currently, high-symptom proportions are very low, because patients are categorized as high-symptom based on being stepped up to the SMH team that the user selected. In real life, many patients are stepped up to other SMH teams, so categorizing those patients as low-symptom seems wrong. We propose defining high-symptom patients as those who are above 50% on the symptom distribution for this team. There would not necessarily be the same number of high- and low-symptom patients, but a new patient to the team would be equally likely to flow into either stock. This seems easier to explain than the current state, where we sometimes have to explain percentages as low as 2%. This is very little work on both the data and the model side. This change would necessitate changes on the sim UI - both i text and removing the symptom proportion entries in the team data table.

  2. Floors should be re-examined In light of almost all engagement durations being below the floor for recent data pulls, it seems like we should re-think the clinical floors and how they map onto the transfer logic. We went back and looked at meeting notes, and found that the floors may have been implemented incorrectly to begin with - the 24 week time to stabilize was applied everywhere. Also, we think PC/PCMHI should probably be exempt from most or all of these floors, because they don't necessarily perform the same clinic activities i.e. they're not a place where patients stabilize. Andrew will try to pull some example cases for us to think through the interaction between the clinical floors and the transfer algorithms. This is very little work for the data and model side. This would necessitate some sim UI i text changes.

  3. Use episode count parameters Currently, we are in danger of using medians that have been calculated from very few data points and then applying them to all patients in a setting, potentially distorting the patient outflows. We propose pulling the episode counts along with the engagement durations, which Tom can use in the model to weight the flows to better fit to the team. There are some good reasons to proceed with this change, but it would take a significant amount of extra work on both the data and the model side. The sim UI shouldn't be affected.

lijenn commented 3 years ago

@lzim I think the changes/addition just need to be documented in the see/say/tms video documents, is that right @jamesmrollins?

Forgive me as I know it had been a while since we looked at this and I might need a refresher on the changes. I believe we also think this would be a good use case to test out the linters as you're already doing right now with the see/say guides.

branscombj commented 3 years ago

@lzim I think the changes/addition just need to be documented in the see/say/tms video documents, is that right @jamesmrollins?

Forgive me as I know it had been a while since we looked at this and I might need a refresher on the changes. I believe we also think this would be a good use case to test out the linters as you're already doing right now with the see/say guides.

I think @lijenn is right - @dlkibbe and I need to figure out what changes to guides are required due to these model changes and then get with @jamesmrollins to implement through documentation. Is that right?

jamesmrollins commented 3 years ago

@branscombj - Yes, we need to get together. While the linter and related actions work just fine, our documents are not ship-shape and will require a lot of editing to clear the linter checks. I am working on those as we speak and will confer with @dlkibbe next week to move the process along.

FYI: @lzim @dlkibbe @lijenn

lzim commented 3 years ago

@jamesmrollins Is this complete?

@lijenn @staceypark - I've added this as blocking #1670 due next week, which means the guides need to be ready this week - edited title accordingly.

jamesmrollins commented 3 years ago

@lzim

As far as my contributions - yes. The actions are complete and ready for use. I can be available to provide direct help as people become familiar with how to operate them. The instructions are in the Team PSD file. I have completed a directory scan of all MTL Blue and Red files.

lzim commented 3 years ago

File needed for DEMO and mtl.how/blue that has valid SP data.

Correct file: "100a1_abc_teama_All_2020_03_10.xlsx"

Next Steps:

[ ] "Get Team Data Table" make sure naming are lower case (Anthony create a new issue card #1736) wk1 mar_epic

[ ] Put this file mtl.how/blue (James)

[ ] Change file to use lower case (James)

[ ] Lower case for team, so they can find it. How do find out what team is using this file? (James).

jamesmrollins commented 3 years ago

Discussed at 2/22 WGLM. This issue was dropped.

Changes that need to be made:

lijenn commented 3 years ago

Hi @jamesmrollins, just making sure, the "!" should be in both the model diagram and the team data table, right? Does that mean there should be an (!) here in the diagram since there is one in the team data table or did I get this wrong? image

jamesmrollins commented 3 years ago

@lzim @staceypark @anthonycpichardo @lijenn

Reference: #1746

jamesmrollins commented 3 years ago

Hi @lijenn - reference your question above about "!" icons in the model diagram. I cannot recall exactly what was decided and don't see anything in the conversation record. I will have to go back and carefully review design docs and see. Since it is not there, I suspect it didn't make the cut, but I will confirm.

jamesmrollins commented 3 years ago

Hi @lijenn. Yes, the "!" icon should be present for team data variables indicated in the table AND in the diagram. I don't believe they are indicated with team data variables that are also controls, as they can be set higher than the zero the model returns when floor counts aren't met. Moreover, the "!" icon sitting outside the control icon may look a little out of place. If I'm wrong about that, we can likely get that fixed in short order. Let me know . . .

staceypark commented 3 years ago

guide updates for #648

Guide language workshopped during the meeting

Our goal is to make the learners aware when low counts used for estimates. A key systems insight for facilitators is that teams will always find the highest leverage change focusing on where the patients are (i.e., stocks or flows).

Sometimes we may have estimates derived from very few observations. We decided to set infrequently observed values to zero to avoid inflation of rarer episodes of care.

Poor inferences are possible using estimates when the count is low and the duration is low. If the values are <1% of the total counts of episodes of care in the team, and they are retained in the model, it will inflate rare episodes.

Here is example competing for the same outflow of patients from PC/PCMHI PC/PCMHI to GMH

  • Episode count is 31 and duration is 30 weeks. This is most common step-up path.

PC/PCMHI to SMH

  • Episode count is 1 and duration is 14 weeks. This step-up engagement pattern was only observed once.

Because the flow is based on the engagement duration (weeks), below is an example 30:1 difference in episode counts, but the difference in engagement duration is actually only 2:1 (30 weeks to 14 weeks).

Retaining this parameter to estimate these model outflow (pts/wk) over the course of a two year experiment timeline would exaggerate the number of patients who step up from PC/PCMHI to SMH from 1 to 20 and reduce the number who step up from PC/PCMHI to GMH from 30 to 10.

If we set it to zero it won't include this rare flow in the basecase. Because the outflow that is infrequently observed is set to zero, it will slightly inflate the other stock outflows, but this is better than extreme exaggeration of an uncommon episodes of care.

However, the Measurement Based Stepped Care for Suicide Prevention module includes low base rate patterns for high risk flag patients.

The outflows from the HRF Patients stock are to residential

Sim UI - Setting View [CURRENT IS]: PC/PCMHI - Residential or Unflag (2 outflows) GMH - Residential, GMH or Unflag (3 outflows) SMH - Residential, GMH or Unflag (3 outflows) - no change

Setting View [CHANGE NEEDED IS]: PC/PCMHI - Residential, GMH or Unflag (3 outflows) GMH - Residential or Unflag (2 outflows) - likely to keep their HRF flag patients SMH - Residential, GMH or Unflag (3 outflows) - no change

Inpatient was not included as an outflow from any setting view because the patient never really transfer fully to the inpatient setting, they are still typically managed by the team coordinating their treatment before the inpatient stay and after discharge from an inpatient stay.

The episode counts for HRF patients are always likely to represent a low percentage of the total episodes of care in the team.

We use the same logic for parameters that control the outflows from the HRF patients stock. These parameter estimates are set to zero when they are < 1% of the HRF episodes of care, and not when they are < 1% of the total episodes of care in the team.

  • Tom is going to test this and check the <1% threshold.
  • James will review the Sim UI model diagram issues identified above and update his estimates.
  • Stacey will add this dependency to the document_tracker.