standardization

[lzim]

Standardization Committee

Standardize all Variable Names, units and Descriptions/Definitions

• data UI
• Vensim models
• sim UI (MTL 1.6)
• R code
• TeamPSD scholarship
• MTL Live, MTL Facilitate, and MTL Video Resources

[lzim]

Next Steps:

1. Debbie and Jane will work to incorporate edits to the master crosswalk file here…: https://github.com/lzim/teampsd/tree/master/sim_ui_workgroup/crosswalk

2. …that we made since Friday in which the data UI is primary, and which Andrew posted here: https://github.com/lzim/teampsd/tree/master/quant_workgroup

3. Since many workgroups need it, and since the sim UI is tertiary to the data UI (data UI is primary to everything for the relevant ‘team data’ variables), and to the Vensim models (Vensim secondary to data, MTL sim UI follows, and all TeamPSD scholarship), please move the most current “Master Crosswalk” to a more general folder of the Team PSD Master “TeamPSD > Resources here: https://github.com/lzim/teampsd/tree/master/resources/master_crosswalk a. I started a “readme.rmd” file for this folder, which we can revise.

Plans for Standardization of Vensim models through Github:

1. Please add any issues or items that need review at tomorrow’s 9AM MTL Launch meeting, you can view and add to the current agenda here: https://meet.lucidmeetings.com/meeting/191647

2. During tomorrow’s meeting, we will show the Standardization Committee a proposed plan for reviewing/revising the Vensim models using GitHub pull requests for code review, like this example here: https://github.com/lzim/teampsd/commit/5eac487c334df15f0c160ae82b7a222cf92dc345#comments

3. I’ve started a new issue for assigning and tracking the standardization issue, which will cover tomorrow. You can find it here: https://github.com/lzim/teampsd/issues/64

4. I’d like to close the original “Master Crosswalk Issue” opened back in May, but it may be helpful to some folks to review the discussion thread there before I close it from our TeamPSD Issues list: https://github.com/lzim/teampsd/issues/54

[branscombj]

Today's meeting: MTL Launch or should it focus specifically on MTL Video as it's called in Lucid?

[lzim]

Jane,

I originally asked Stacey to name this meeting the following way: “MTL Launch” because it is the meeting where all our developers for the national launch come together once a week.

Then, add the detail to clarify part of the agenda: “MTL Launch/MTL Video” and “MTL Launch/MTL Facilitate.”

Unfortunately, that was not done originally, so there is a mix of labels in our invites and calendars. Once something goes out, it has a legacy in a variety of (sometimes unfortunate) forms. In Stacey’s absence, I’m working to fix confusing issues like this.

By “MTL Launch” we mean that we can always address issues related to our first launch “MTL Live” with Elizabeth and/or Justin from EES, and James from Takouba. This is our top/first priority and will typically require the majority of our agenda.

Then, we will alternate having point people from our other two EES projects in attendance in the following ways: 1st/3rd - MTL Live/MTL Facilitate (Fawn to join) 2nd/4th - MTL Live/MTL Video (Ann to join)

By no means are we to focus on just MTL Video at this meeting today with this other priorities taking precedence. But, our MTL Video producer Ann Hier plans to join at this regular interval for part of the agenda,as she learns more about the project. We can cut her loose to cover our other content.

I apologize about the confusion and will work to root it out as we go forward.

Thanks!

Lindsey

[branscombj]

Got it, thanks.

[lzim]

Hi Everyone,

Here are some next steps Jane, James and I decided on today regarding standardization: Next Steps for Standardization

1. James will draft a prototype of a "master file" that brings into conformance the work Jane did and his Google spreadsheet for developers - the goal of this task is to have one "master" that meets the needs of sim UI development, while also making it possible to extend to cover all TeamPSD use needs (every instance of every variable outside the needs of the sim UI).

2. Lindsey and James will keep in touch with each other on Friday 26 July 2018

3. Lindsey, Jane, Debbie, Tom and Savet will work together via GitHub to establish how to make decisions about standardizing variables names in Vensim using GitHub.

[jamesmrollins]

It kind of dawned on me that the table that Jane and Debbie prepared earlier is really from a visual perspective. While a label may show up in in several locations visually, it may indeed be driven by different variables functionally. Such is the case for example, in line 26 for Referral Rate, and line 33, User Defined Appointment Supply. Both have labels that are the same visually, but are pointed to different vensim Variables. This probably easier to explain in person . . .

1. I have tried very hard to maintain consistency on the basis of the original location of the variable name, such as from the model parameters file or the vensim model. I have mapped each variable according to what it supports, while mapping it visually as far as it goes across the various platforms.

2. I added a Standard Units column to provide a place to put the standard label. Exceptions are noted in the comments at the header of the column.

3. Feedback loops are listed on the bottom of the sheet. In the units column, I indicated the direction and type of the feedback loop (B-CCW means Balancing Counter-Clockwise).

4. The table also includes variables and data that are not used in other platforms, but are included for reference.

5. I have provided comments with each header to explain what is underneath the header.

6. There is a legend on the bottom of the sheet that explains the color codes.

7. With the new search and filter capability we are building into all the drop down menus in Ver 1.52, I am not sure we can keep the default chart order any longer. The new default may be alphabetical. I will ask my developers for information on this. If the default chart order can be maintained, then I think we should provide the chart default order in parenthesis, immediately preceding the chart name (as in Jane and Debbie’s version).

8. Not sure what to do with the Reveal Complexity or Display Patient Cohort subheadings.

9. This took about 3.5 hours to compile in our most simple model. There is likely 3 days of solid work here, as all of this has to be walked by hand across the platforms.

10. An Excel spreadsheet is loaded on GitHub in the Resources Folder, sub-folder Master_Crosswalk.

11. The Google version is here: https://docs.google.com/spreadsheets/d/10jA9TWlrZH0nSYjl8asHRVvHmX1H5lghCHqsLe8LNcs/edit?usp=sharing

[branscombj]

In re: the part above that I bolded, James, as you have pointed out, there may be places where we decide to accept a slight inconsistency in names for a specific variable across platforms (contents of a row in the crosswalk not all looking identical). There are also places in which a transformation of a variable or variables occurs from one platform to another – in which case I’d argue that the transformed variable is a different one and should have a different name (different row) from the source one(s). Unless I’m missing something, there should not be a case in which label A in one platform points to variable B in another platform that also has a variable called A (a specific quantity appearing on two rows.) It should be possible to have columns for each different place variables appear in MTL, and to line them up horizontally when they mean exactly the same quantity, so that where they appear in more than one column (which they won’t in every case), we can standardize and track them.

Does this make sense?

[branscombj]

So, I need to know what GHPC's responsibility is for crosswalk development going forward - other than participating on the standardization workgroup to help make standardization decisions. If the work James did last night moves us closer to having what the project needs than the latest GHPC workbook, then he's probably the best one to carry it through for the other models. We just need to be sure that we pick up in that document the changes that have been decided recently, since we suspended our efforts on the tasks assigned to us above for the crosswalk pending James's new prototype.

[jamesmrollins]

It does – but during iteration B, the team broke the convention you state below. I warned that design features in the Sim/UI the team desired, would create the inconsistency you cite below. However, we did come up with a work-around that works functionally, which is reflected in the matrix.

James Rollins Managing Partner Takouba www.takoubasecurity.com (425)919-5153

From: branscombj notifications@github.com Sent: Friday, July 27, 2018 9:32 AM To: lzim/teampsd teampsd@noreply.github.com Cc: James Rollins james.m.rollins@takoubasecurity.com; Assign assign@noreply.github.com Subject: Re: [lzim/teampsd] standardization (#64)

In re: the part above that I bolded, James, as you have pointed out, there may be places where we decide to accept a slight inconsistency in names for a specific variable across platforms (contents of a row in the crosswalk not all looking identical). There are also places in which a transformation of a variable or variables occurs from one platform to another – in which case I’d argue that the transformed variable is a different one and should have a different name (different row) from the source one(s). Unless I’m missing something, there should not be a case in which label A in one platform points to variable B in another platform that also has a variable called A (a specific quantity appearing on two rows.) It should be possible to have columns for each different place variables appear in MTL, and to line them up horizontally when they mean exactly the same quantity, so that where they appear in more than one column (which they won’t in every case), we can standardize and track them.

Does this make sense?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/lzim/teampsd/issues/64#issuecomment-408471573, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcvHMaEZU6kol5XRGdlfCj6apofIm-jkks5uK0CPgaJpZM4Vg3m7.

[lzim]

Happy Friday Everyone!

James and I met today and made the follow decisions for next steps. Please review the details below.

NEXT STEPS:

1. **MTL 1.6 Standardization is top priority (Tom, Jane, Debbie, Lindsey, Andrew, James and Savet are on the standardization committee). See the following intermediate deadlines. Need to finalize all variable names via GitHub Review of Vensim models & Master Crosswalk table. Discuss any questions or concerns about the variable names to meet the following interim deadlines for each module.

   • CC - Monday 27 Jun 2018 - COB
   • MM - Wed 1 Aug 2018
   • Psy - Fri 3 Aug 2018
   • Agg - Mon 6 Aug 2018 - COB **NOTE the Takouba/Forio coding/development cannot commence until standardization decisions are reached, and we need all final, so that we have a standardized set of Team PSD and MTL resources: data UI, Vensim models, sim UI, and R code for our MTL Facilitate Pilot 4-7 September with the OMHSP TAS.

2. **James will try to inventory the variables in each model (CC, MM, Psy and Agg) that require two rows to explain their logic (per the thread above with Jane).

3. **We decided to add a column A to the master that will enable sorting by the type of variable:

   • MPO = Model Parameters Only
   • VO = Vensim Only
   • CF = Cross-functional (variables used by all platforms)

4. **Standardization Workflow: The most current version of this "master" table should always be posted to GitHub here: https://github.com/lzim/teampsd/tree/master/resources/master_crosswalk

5. **We'll work on using pull requests to work on the crosswalk master.

6. **James will drag

7. Once standardization is complete, weed to have the standardization committee members review the variable "i" information in the Vensim model comments in GitHub, and we will begin to populate the .csv document that James will use for programmers to provide this information in the sim UI.

Thanks All, have a great weekend!

Lindsey

[jamesmrollins]

Hi Jane, I reread your feedback and now it makes sense. I think I have accomplished that with respect to the variable that requires an exception. When I get into the nitty-gritty of completing these crosswalks, I will formulate some kind of cross-model consistency test.

Thanks, James

[jamesmrollins]

Team After a review of all the models, I only come up with one variable each in CC, MM and AGG that require an exception to our rule, regarding the consistency of labels between the Model Parameters File, the Model and the Sim/UI. The exceptions are noted below:

Model/ Vensim/ Sim/UI CC/ User-defined Referral Rate/ Referral Rate MM/ User-defined Referral Rate/ Referral Rate AGG/ User-defined Referrals to Team/ Intake Evaluation Rate

I will upload the latest xlsx file.

Thanks, James

[lzim]

Thanks @jamesmrollins, with the exception of the three instances you list above, we will have a single row for each variable wherever it occurs in our MTL resources.

When I download the latest Excel file by clicking "view raw" here: https://github.com/lzim/teampsd/tree/master/resources/master_crosswalk I'm getting an Excel dialog with a request to repair the Excel file, any ideas about why that may be between Google > GitHub > Excel for Mac?

If the repaired Excel file has retained all the information, then does the "Model-Pop-Up-Information" tab refer to "i" information for the sim UI? If so, did you draw that from Vensim for all four models? Or, did you include only the comments for Vensim variables that are also in the sim UI?

Four our new column A - it looks like CC has MPO, VO and CF variables. So, everyone (@branscombj , @TomRust, @dlkibbe) can review to make sure we can finish CC standardization by COB Today, right?

Thanks again!

Lindsey

[jamesmrollins]

@lzim - This is a repair issue between Google Sheets and Excel for the comments that are in the headers of the CC sheet. If you allow the repairs, it should have no effect on the file. I have to do this also when I load the file into my PC version of Excel.

Yes, the model-popup-information is for feeding the "i" information into the Sim/UI. The "i" information for team data is revised - final. Experimental Variables "i" information comes from the Sim/UI, and the Model Variables "i" information comes from the Vensim comments field for each variable, and need to be reviewed.

[lzim]

Okay, thanks James!

[jamesmrollins]

Team, I have just uploaded Master_Crosswalk_Table_v2_20JUL18. This includes a complete listing from the Model Parameters file and the Vensim MM model. For those variables that have not had "i" information to this point, I copied Tom's comments from the comments section of each variable identified in the list to give us a starting point. I will move on to PSY next.

I am still not completely sure about how to use GitHub, so I hope my file uploading isn't disrupting anyone's workflow.

Cheers! James

[lzim]

Standardization Committee Decision We will not label flows with "CC" and "MM" etc. It will just be a standard for all modules, e.g., "starting rate" and "ending rate" We will label the stocks with "CC" and "MM" for patients and appointments, e.g., "patients in CC" We will not label median engagement with any service specific information, e.g., it will just be "Median Engagement" Use the following file naming convention so they are sortable "year_month_date_time" or "yyyy_mm_dd_tttt" (NOTE: most files will not include time). Here is an example: "2018_08_01_0800"

• James is keeping the most current version of the Master here: https://github.com/lzim/teampsd/tree/master/resources/master_crosswalk
• The column labeled "Standard" is the Standard definition for that variable. If the variable in the same row in your MTL resource is purple, then it means you need to update your resource to make it consistent with the standard.

@branscombj, @dlkibbe, @TomRust, @saveth, @holbrooa, @staceypark

[jamesmrollins]

@TomRust @lzim I am not sure what these variables in AGG map to in Vensim. If we want them to have an "i" explanation, I need to have a variable to map them to.

[jamesmrollins]

@TomRust @lzim I am not sure how to map these two feedback loops, "Start Delays Affect Treatment Decisions" and "RVI Delays Affects Treatment Decisions." Furthermore, the Vensim model appears to have two more feedback loops than the SIMUI.

[lzim]

Hello Standardization Committee,

@saveth and I worked on standardization today, and found the following remaining inconsistencies:

In the ModelParameters.xlsx file cells B15:B32 - Engagement Attributes for patients after their first 3 months:

1. Shouldn't they all read "who return" to be parallel throughout and consistent with Sankey?

2. They were all changed to read "median" and we need to clarify our thinking again, even though the committee discussed it

   • I believe it means that the group is defined by the 1st quartile, mean, 4th quartile
   • Then it is median for that group in appt/wk (for cells B15:B23
   • Then it is media for that group in wks (for cells B24:32) Is that right? This is definitely still confusing in the sim UI, and it is a confusing definition.

3. Tom also noted that there is a definition that says one year in one place and two years in another. As of last week, the most correct definitions from the Standardization Committee for the ModelParameters.xlsx file is here: https://github.com/lzim/teampsd/tree/master/quant_workgroup Andrew posted it 8 days ago, however we made the changes I describe in this post since then.

4. Savet will update the Team Data R Code to say "Median Engagement" for CC and MM (removing the "in CC" or "in MM" service or module from the name.

5. She will also use the now standard "starting rate" for all modules, CC, MM, Psy and Agg (see does not have ending rate)

NOTE: Savet plans to use columns B and C of the Master Crosswalk to stay current, but her R code will always be reading directly from the "ModelParaeters.xlsx" file.

Thanks Everyone!

Lindsey

Note: there is also a lot of discussion of the .mdl standardization here: https://github.com/lzim/teampsd/pull/67

@branscombj, @dlkibbe, @TomRust, @jamesmrollins , @holbrooa, @staceypark

[jamesmrollins]

Master Crosswalk updated to include AGG model, sans the comments fields for variables not associated with the Model Parameters nor the SIM/UI. Since the team is editing comments in GitHub, it makes better use of time to just wait and then update the crosswalk afterward.

The Master Crosswalk is also updated with most of the PSY information, but I need help making sure I have properly mapped the variables in the Team Data Table. Then I can finish adding "i" information. I also need to add standard units.

One issue I have discovered, is that we don't have a standard unit in PSY for "Points" used to describe the EBP Standards (Bronze, Silver, Gold). I proposed "pnts" in the meantime. The only place I see that we really refer to these is in charts in the SIM/UI. I am not sure if we would arrange experiments around these variables.

Thanks, James

[jamesmrollins]

@lzim - Lindsey, if possible I would like to cross-level information with you tomorrow. It shouldn't take long. I think I am still have some lingering confusion with the process; especially with respect to model update procedure. We are now delayed two days and I would like to resolve straight-away if possible.

Thank you, James

[lzim]

Hi Everyone,

I just reviewed the CC pull request from Tom, made edits in Pull Request #80

Thanks!

@branscombj @dlkibbe @jamesmrollins @saveth @TomRust @staceypark

[lzim]

@jamesmrollins Have you done any more work on the master crosswalk, or should I look at the one you posted to master yesterday?

Thanks!

[jamesmrollins]

The Master is up to date with version 9. Will likely post version 10 tonight.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Lindsey Zimmerman notifications@github.com Sent: Thursday, August 30, 2018 10:57 AM To: lzim/teampsd Cc: James Rollins; Mention Subject: Re: [lzim/teampsd] standardization (#64)

@jamesmrollinshttps://github.com/jamesmrollins Have you done any more work on the master crosswalk, or should I look at the one you posted to master yesterday?

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/lzim/teampsd/issues/64#issuecomment-417411297, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcvHMRuebPbUXUfu2dkraxNSOb8gVVIpks5uWCecgaJpZM4Vg3m7.

[lzim]

I'm going to close the standardization issue at v10 with the MTL 1.6 release today. We can create a separate issue as we work on dynamic "i" information.

Thanks for all the great work on this @branscombj @staceypark @dlkibbe @jamesmrollins @saveth and @TomRust !!!