Task: Automatic Model Update Process

[jamesmrollins]

Problem The current model crosswalk table is a spreadsheet that lists the standard label for variables and attempts to link the standard name to the actual labels used on other resources (i.e., Team Data Table, Vensim model). Standard labels can be difficult to implement because the Sim UI often needs unique labels for the same function in order to execute code. Furthermore, most of the models were developed prior to the establishment of the code; therefore, are not standardized. Finally, there is no procedure documented for editing, updating or quality checking label crosswalks creating a single point of failure.

Mission Identify and document the current standing operating procedure for documenting resource labels. Determine if there are ways to automate the functions in order to reduce errors and to ease demand.

[lzim]

@jamesmrollins @anazariz

Note - We say in our grants that we will document our Vensim DSS models using SDM-DOC Goals: 1) Error-checking and Standardization 2) Model documentation for review by scientists and others

1) https://www.systemdynamics.org/SDM-doc 2) http://wayback.archive-it.org/10432/20181121203235/http://lm.systemdynamics.org/tools/sdm/Handbook%20Model-A.html#a105

Cross-ref: #784 Cross-ref: #886 - How can we get to a more streamlined file upload for Team Data Tables? This came up during our 4/1 Support Workgroup Leads meeting. BROADER GOAL: Get rid of anything that manually has to be done with the Epicenter platform.

[jamesmrollins]

Requirements Testing and Validation Checklist

Problem Statement: Use of labels between Team Data files, Vensim model files and Sim UI labels is very difficult to standardize and often results in errors. The standardization of labels, wherever possible, aids in mapping variables across the various platforms. In the model-Sim UI validation process, values from the simulation are compared, by hand, against values returned from a client session of the model.

Requirements

1.0 Develop a database routine that will identify exceptions between two model versions and the Sim UI label map.

• [ ] 1.10 The database routine will read the output .mdl (edited 04/08/2020) or text version of a Vensim model, that describes variables, their units of measure, minimum and maximum values, default value and associated comments and parse into a database. This will be the database of record. It will contain all variables in a given model.

• [ ] 1.20 The database routine will identify exceptions between two models and generate an exception report. The exception report will report any differences between the two files, including text differences between comments fields.

2.0 Develop a automated disposition process that will record inputs and provide a record of accountability for actions in the record.

• [ ] 2.10 The exception report will drive a disposition process where a Sim UI Workgroup Lead will determine if the excepted variable is cross-functional (CF) and used between model and the Sim UI or the Team Data Table, the model or the Sim UI.

• [ ] 2.20 The disposition process will provide a means where the Sim UI Workgroup Lead can provide a Sim UI label name (following naming convention identified in the algorithm).

  • [ ] 2.21 The disposition process will provide a means where the Sim UI Workgroup lead can provide a Sim UI label name and location grid for Team Data related variables.

• [ ] 2.30 The disposition process will provide a means for the Sim UI developer to verify the proper use of the label name, its units of measure, its minimum and maximum values and default values.

• [ ] 2.40 The disposition process will enable the Sim UI Workgroup Lead to verify changes and record date/time of disposition closure.

3.0 Develop a process to host an SDM-DOC.html file as a webpage, under a "technical specifications" heading on mtl.how/demo. (see edit below from Support Workgroup Meeting 4/8/2020; see below).

• [ ] 3.1 Devise a process to update a model SDM-DOC.html file and upload into Epicenter.
• [ ] 3.2 Devise a CSS Design file that will provide table and information formats consistent with VA style guide.
• [ ] 3.3 Create an .html SDM-DOC file for each model and upload into Epicenter
  • [ ] 3.31 CC
  • [ ] 3.22 MM
  • [ ] 3.23 PSY
  • [ ] 3.24 AGG
  • [ ] 3.25 SP
• [ ] 3.4 Create a Model Documentation link or button under a "Technical Specifications" heading in MTL/DEMO.

Algorithm

Wire Frames

1. MTL/DEMO
2. html table formatted with VA standards (Myriad Pro, colors)

[lzim]

@jamesmrollins

Decisions at 4/8/2020 Support Workgroups #1220 - Requirement 3 - Provide Model Documentation for Scientific Audiences

Completed Verfication of SDM-Doc: James has taken the SDM-DOC java app and was successfully able to generate a .html from the MTL 2.0 .mdl Vensim DSS.

1. Decision James will edit this Issue Task Card for putting this CSS Design file.
2. We develop a CSS that specifies our VHA Graphics/Style requires
3. Host the .html in our VA style on mtl.how/demo - "Model Documentation" in "Technical Specs."

[staceypark]

@jamesmrollins Thanks for moving this to the upcoming epic following the Monday WG Leads call. Please make sure to also update the milestone and epic on the right-hand side.

[lzim]

Discussed at #hqhuddle on 4/24/2020

1. As we develop our improved - "Master Crosswalk" process (Protocol)
2. Need to coordinate with cross-ref: #1367 (SQL Style Guide - Ash) and #1366 (R Style Guide - Anthony).

Master Crosswalk Goal: Is that standards are developed and every instance of a Team PSD or MTL variable instance, data definition, code base (e.g., interdependent .Rmd, data and .R; SQL joins etc.) in a way that is consistent across Team PSD workgroups.

Next Steps:

1. @anazariz and @anthonycpichardo we re-review @jamesmrollins algo above and requirements above, learn a bit more about the old master crosswalk process and improvements we seek, and work to develop SQL and R standards that are consistent with a comprehensive Team PSD process.
2. Additional SQL and R recommended style/standards will be presented for review at the Support Workgroups Meeting on Wed 4/30/20.

[lzim]

Team GitHub Repos **Cross-ref

1192

1220**

Lindsey will put this together with visual icons for Team PSD Values and Team PSD Principles.

It would be great to get a Visual Map that also shows how these go together.

Team PSD Scientific Values guide additional Participatory and Open Science principles:

• Participatory Research encourages us to co-create our scientific research. Therefore...
• We share decisions, which requires a high level of documentation
• We seek greater equity among partners in how collaborate, which requires responsive pivots with new stakeholder inputs
• We strive use transparent and accessible processes and platforms, and develop transparent, accessible resources.

Team PSD Project Management Principles

• NOTE: Team PSD Scientific Values guide how and why we synthesis the approaches below

Team PSD integrates Waterfall principles because:

• We are a research team that cannot deviate in quality, scope, timeline or cost from our thoroughly developed and vetted research plan
• Therefore, there are several well-defined scientific requirements where change cannot occur - unless the PI and co-I senior scientists advise an operational definition or implementation detail is within the scope of the federally funded and registered, scientifically peer-reviewed, and VA ORD and Stanford IRB reviewed procedures.

Team PSD integrates Agile principles because:

• Our research team’s method is a an implementation of participatory system dynamics in a national production environment.
• These methods historically have iterated in-person locally.
• Our innovation is local priorities, data and insights at national scale, made possible using online platforms and virtual processes.

Team PSD integrates Waterfall and Agile principles because:

• We are using a funded a priori research design (Waterfall) to study an implementation science (Agile) problem.
• Producing generalizable knowledge requires implementation scientists to follow and report publicly progress completing a rigorous, replicable plan (Waterfall)
• However, the context is not a tightly controlled laboratory, rather we are working with operational partners to implement our innovation the dynamic context of real world healthcare operations (Agile)

Team PSD integrates Lean principles because:

• Team PSD processes are assessed for continuous improvement (kaizen/muda elimination)
• Team PSD communication is designed to reduce rework (muda/waste)
• Team PSD encourages minimum viable products to find and synthesize team and partner expertise (just-in-time)
• Team PSD platforms are designed to increase visual production controls (kanban/signboard; andon alerts)
• Note: VA uses Lean as a primary approach to healthcare quality improvement.

Team PSD integrates Scrum principles because:

• We need to be able keep work produced by our transdisciplinary research team on the same page with our cross-functional team.
• Therefore, we use sprints (epics/milestones)so that the team priorities can be aligned/re-aligned efficiently.
• We also use workgroup leads (scrum masters), workgroups (owners) and workgroup meetings to benefit from the efficiency of divvying up/delegating, while also identifying dependencies and remove blocks.
• We use GitHub/ZenHub and daily huddles to assign, scope, prioritize, manage and review our capacity, requirements, estimates - this includes Project Kanbans, Issue Cards, Pipelines & Reporting.

Team PSD Pain points we are working to better address:

• Need tight collaboration without unnecessary overhead, so culture of participation is maintained
• Need rapid iterations that prototype, check early, and synthesize input from key stakeholders as efficiently as possible, so we aren’t pulled in wasteful directions or adding unnecessary rework
• Need a stable set of principles and processes, so our production platforms are stable at scale

Continuous Collaborative Iteration Cycles (e.g., “DevOps”)

• SOP > Workgroup Meetings > Monthly Priorities > Requirements gathering standardization for features
• Tools for team members to understand cross-functional “stakes” efficiently
• “Continuous integration” to avoid “merge conflicts” - Quotes are meant to clarify that we have this problem at a communication and conceptual level, not just a code level.
• Ideally...testing is what would be increasingly automated
• • REALLY need continuous documentation and training.

VA GitHub Repo

• May move our Repo here over time....
• See if we have or can get Git on any machines (didn’t used to be able to, but maybe can now)

Ad Hoc vets.gov - benefits tracking NICE Examples for #1220 https://github.com/department-of-veterans-affairs/vets-website

https://github.com/department-of-veterans-affairs/caseflow

va.gov https://github.com/department-of-veterans-affairs/va.gov-team

NICE Examples for Team PSD Manual: Plain English for #1192 https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/platform/working-with-vsp/orientation/repo-guidelines.md

With Maps https://github.com/department-of-veterans-affairs/lighthouse-facilities

Git for Version Control

https://towardsdatascience.com/getting-started-with-git-and-github-6fcd0f2d4ac6

https://rogerdudler.github.io/git-guide/

DevOps for Automating Team Dependency Flows and Reducing Errors and Rework

• Develop code tests
• Automated deployment
• Automated testing of pull requests for code review

[lzim]

Discussed at Support Workgroup Meetings on 4/29/2020

1. Talked about the interdependencies of the requirements above and the 4/24/20 #1367 (SQL Style Guide - Ash) and #1366 (R Style Guide - Anthony).
2. Lindsey added information about integrating with our GitHub Workflow and linked to #1192.
3. We considered "refactoring" or downstream corrections of all instances, which seemed like a new requirement.
4. Follow-up w/James next week.

@anazariz @anthonycpichardo @jamesmrollins @staceypark

[lzim]

BTW: Thanks to @anazariz for encouraging me to get the Team PSD Manual "Project Management" (Column A) information out there in terms of how/why we enlist the synthesized project management approaches we do. 🏆

• I'll create a .md for this information for editing/review, and ultimately knitting in the manual.
• This cross-walks with #1192, #921 and likely more... 📚

[lzim]

@jamesmrollins Is the algorithm diagram in LucidCharts?

I was thinking we could review it further with our insights from yesterday. I also see that next steps from our 4/24/2020 post said that 1) Anthony & Ash would review the algo diagram, and 2) the current Master Crosswalk.

@anthonycpichardo @anazariz Have you seen the current master crosswalk?

[lzim]

Happy Friday Support Workgroups! It’s May! 🌳 🌻 ⛰️ 🏖️

@staceypark @jamesmrollins @anazariz @anthonycpichardo

Great work to keep this moving forward! 👏

• James’ work on the definitions and algorithm, and Ash and Anthony’s MVP really seemed to move us toward a more aligned team mental model. THANK YOU 💯
• As team lead, here is our summary and next steps based on Support Workgroup quant_workgroup and sim_ui_workgroup meetings on Wed & Thurs.

BLUF:

1. No changes to purpose, problem definition and current requirements (3/19 & 4/7).
2. But, requirements for automated integration and standardization of “... every instance of a Team PSD or MTL variable, data definition, code base...[to stay]...consistent (4/24) requires increased use of code and the GitHub platform for efficient version control.
3. Setting this up requires training/team learning.

NO CHANGE TO:

• Purpose: “Create a Master Crosswalk Process”
• Problem: “current model crosswalk table is a spreadsheet” “not standard” “error/failure prone” and from requirements...”by hand.”
• Functional Requirements:
  • 1.0 Develop a database routine that will identify exceptions between two model versions and the Sim UI label map.
  • 2.0 Develop a automated disposition process that will record inputs and provide a record of accountability for actions in the record.
  • 3.0 Develop a process to host an SDM-DOC.html file as a webpage, under a "technical specifications" heading on mtl.how/demo. (see edit below from Support Workgroup Meeting 4/8/2020; see below).

ADDITIONAL REQUIREMENTS:

1. 4.0 Comprehensive integration of Team PSD code at mtl_code (e.g., .md, .R, .sql, .html, Vensim) (4/24 Cross-Ref #1366, #1367) - requirements above are very sim focused
2. 5.0 Integration of “every instance” should integrate with Team PSD GitHub Workflow (4/29 Cross-Ref Team PSD Manual #1192).
3. 6.0 Automated testing should include re-factoring for automated Team PSD/MTL-wide version control.

NEXT STEPS: GitHub TRAINING PLAN (6 mini courses to learn)...

GitHub: YAML Configuration, Continuous Integration, Delivery & Documentation

GitHub Training

TRAIN 1: GitHub Apps - Automate Repetitive Tasks

https://lab.github.com/githubtraining/getting-started-with-github-apps

We'll answer common questions like:

• What are GitHub Apps and how are they structured?
• How do GitHub Apps fit into the software development lifecycle (SDLC) and the greater ecosystem?
• How do you use the GitHub APIs?
• What are webhooks and how are they used?

TRAIN 2: GitHub Actions - templated workflow with testing

https://lab.github.com/githubtraining/github-actions:-continuous-integration

We'll answer common questions like:

• What is Continuous Integration?
• How do GitHub Actions and workflows enable CI?
• What are build artifacts and how are they used?
• How can a unit testing framework be implemented with GitHub Actions? And when you're done you'll be able to:
• Describe the importance of GitHub Actions in Continuous Integration
• Use and customize a templated workflow
• Create CI workflows that match a team's needs and behaviors
• Use a repository's source code and build artifacts across jobs in a workflow
• Implement a unit testing framework using GitHub Actions
• Create a workflow that runs tests and produces test reports
• Set up a matrix build to create build artifacts for multiple target platforms
• Save and access a repository's build artifacts
• Choose virtual environments for an application's CI needs

TRAIN 3: GitHub Packages

https://lab.github.com/githubtraining/github-actions:-publish-to-github-packages

We'll answer common questions like:

• What is GitHub Packages?
• What role does package management play in the software development lifecycle?
• What is a container, a virtual machine, and Docker? After taking this course, you'll be able to:
• Describe CD and why it is necessary
• Use and customize a repository workflow
• Create CD workflows that matches the team's needs and behaviors
• Use the repository's source code to build artifacts and store them in the GitHub Packages
• Save repository build artifacts
• Access saved build artifacts

TRAIN 4: GitHub Actions - Docker

https://lab.github.com/githubtraining/github-actions:-write-docker-container-actions After completing this course, you will be able to:

• Consume actions within a workflow file
• Create custom Docker container based actions
• Create actions in your programming language of choice
• Leverage an external API within an action
• Set and use input parameters for actions
• Publish your newly created action to the marketplace You'll know the answers to questions like:
• What is a workflow?
• What are actions?
• What is action metadata?
• What is a Dockerfile?
• What are GitHub Actions capable of?
• What programming languages can I write GitHub Actions in?
• How do I publish an action?
• How do I customize my action's behavior based on inputs?

TRAIN 5: Release integration w/our Feature Tracker “SLDC”

https://lab.github.com/githubtraining/create-a-release-based-workflow

• How do I choose a branching strategy?
• When should I use topic branches?
• When should I use release branches?
• How do I use the Projects tab on my repository?
• How do I create a release-centered workflow?
• How do I backport changes for a prior release?
• How do I implement a hotfix?

TRAIN 6: Alternatives: Continuous Integration with Circle CI- automatically test changes made to your project

https://lab.github.com/githubtraining/continuous-integration-with-circleci

Developers integrate code into a shared repository several times a day. With such frequent code changes, how do you ensure your code is bug free? Continuous Integration (CI) is an approach to software development in which tests run automatically anytime code is changed, saving you time and giving your team improved reliability. Continuous Deployment or Delivery (CD) refers to whatever happens after these tests run. If they pass, your new code can be automatically deployed to production.

• Define continuous deployment
• Identify how projects use CI
• Set up continuous integration for one of your projects
• Protect branches based on the result of a build from continuous integration
• Customize builds
• Add tests for your project

Summary and Proposal to use YAML Config as our Process

https://yaml.org/spec/1.2/spec.html

Key Idea 1: Adoption of the configuration file

After developing our standards, when a user creates a new project or it’s on the settings page, we could suggest a functional configuration file with a minimal setup. And making clear where to put global configurations.

Key idea 2: Configuration file and database

The settings used in the build from the configuration file (and other metadata) are human and machine readable across all Team PSD code.

Key idea 3: The build process

• The repository is updated
• Checkout to the current version
• Retrieve the settings from the database
• Try to parse the YAML file (the build fails if there is an error)
• Merge the both settings (YAML file and data)
• The version is built according to the settings
• The settings used to build the documentation can be seen by the user

Benefits of expanding Team PSD use of YAML include consistency with existing Team PSD style, standards, SOP and team Skills:

A. Every JSON file is also a valid YAML file. The design goals for YAML are, in decreasing priority:

1. YAML is easily readable by humans.
2. YAML data is portable between programming languages.
3. YAML matches the native data structures of agile languages.
4. YAML has a consistent model to support generic tools.
5. YAML supports one-pass processing.
6. YAML is expressive and extensible.
7. YAML is easy to implement and use.

B. YAML information is used in two ways: for machine processing, and for human consumption.

• The challenge of reconciling these two perspectives is best done in three distinct translation stages: representation, serialization, and presentation.
• Representation addresses how YAML views native data structures to achieve portability between programming environments.
• Serialization concerns itself with turning a YAML representation into a serial form, that is, a form with sequential access constraints. Presentation deals with the formatting of a YAML serialization as a series of characters in a human-friendly manner.

...Unlike JSON, yaml supports comments

C. Other considerations Ansible Playbooks By integrating their software with YAML, Red Hat developed Ansible, an open source software provisioning, configuration management, and application deployment tool.

• Ansible temporarily connects to servers via Secure Shell (SSH) to perform management tasks using playbooks which are blocks of YAML code that automate manual tasks.

We should work through these trainings on our own and update each other with recommendations in this thread.

• Please note that the examples I listed on Wednesday incorporate several things covered in the 6 mini-course training plan.

NICE Examples for #1220

https://github.com/department-of-veterans-affairs/vets-website https://github.com/department-of-veterans-affairs/caseflow https://github.com/department-of-veterans-affairs/va.gov-team

NICE Examples for Team PSD Manual:

Plain English for #1192 https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/platform/working-with-vsp/orientation/repo-guidelines.md https://github.com/department-of-veterans-affairs/lighthouse-facilities

[anthonycpichardo]

@jamesmrollins Is the algorithm diagram in LucidCharts?

I was thinking we could review it further with our insights from yesterday. I also see that next steps from our 4/24/2020 post said that 1) Anthony & Ash would review the algo diagram, and 2) the current Master Crosswalk.

@anthonycpichardo @anazariz Have you seen the current master crosswalk?

@lzim Ash and I reviewed the master crosswalk that is hosted on OSF when thinking through the parser. Side note: currently, we aren't a part of whichever project the file is hosted on though and needed @jamesmrollins to send it to us. Can we get added to the project?

[jamesmrollins]

@jamesmrollins Is the algorithm diagram in LucidCharts?

I was thinking we could review it further with our insights from yesterday. I also see that next steps from our 4/24/2020 post said that 1) Anthony & Ash would review the algo diagram, and 2) the current Master Crosswalk.

@anthonycpichardo @anazariz Have you seen the current master crosswalk?

@lzim @anazariz @anthonycpichardo Its in there now . . .