search

carpentries / strategic-plan

Repository of public information regarding the progress of The Carpentries 3-5 year strategic plan.
https://carpentries.org/strategic-plan/
Other
5 stars 1 forks source link

Identifying areas for effective and appropriate automation #55

Open kariljordan opened 2 years ago

kariljordan commented 2 years ago

Goal 5 of The Carpentries Strategic Plan reads: Strengthen organisational structure and capacity to be strategic and responsive.

One objectives under this goal is:

  1. Establish workflow and documentation for our essential ongoing processes (e.g. budget creation and review, communication channel use), and identify areas for effective and appropriate automation or other improvements needed to scale support.

This issue tracks the work we are doing to automate internal workflows for:

maneesha commented 2 years ago

~Leaving this as a placeholder for now - I'll come back later to update details.~
EDIT: Details for each project are in a separate comment below (except for the workshop survey results)

Automation projects, April 2020 to present

EDIT 2023-03-13 to change bullet list to checkbox list and add in items created by other Core Team members.

maneesha commented 2 years ago

Automated emails for workshops

Context: Workshop administrators have standard template emails that they send out to workshop Hosts, Organisers, Instructors, and Helpers. These emails include information such as introductions of people serving various roles, reminders about the workshop website and survey, and post-workshop follow ups. The emails all contained the same information, customized to each workshop's details such as date, personnel, and location. Because the emails content was generally the same, it was a tedious task to draft the same emails for each workshop.

New system: Our AMY developer created a system where emails would automatically go out to workshop personnel on set dates relative to the workshop. This ensured that workshop adminstrators did not have to manually generate the text for each email. It also ensured emails would go out at exact dates/times relative to the workshop date because they were automated, rather than depending on the workshop administrator's availability. Read more about the automated emails. Planning work on this system began in 2019, and was implemented in mid-2020.

Next steps: As our operations change, these emails will also have to change. The Infrastructure Team (Maneesha) is working closely with the Workshop Admin team (Sher!) to identify changes needed to these emails based on eliminating the role of Supporting Instructor, running both in-person and online workshops, and implementing the new Instructor Selection project (see below). Automated emails that support other teams (such as Instructor Training) may also be considered in this context. Much of this work will depend on the availability of an AMY developer.

maneesha commented 2 years ago

Automated emails for memberships

Context: When a new membership is signed, the Membership Team has a standard template they use to send to site contacts including information on how to request workshops, instructor training, as well as general information about The Carpentries. Because the email content was generally the same, it was a tedious task to draft the same emails for each membership.

Additionally, member sites would ask about their benefit usage. Providing benefit usage reports on demand becomes another time consuming task. We wanted a system where member sites would proactively receive regular updates about their membership usage to remind them to use their benefits before the membership term ended.

New system: AMY developer time is limited. Knowing this and that unlike workshops, the membership emails can be sent at approximate times relative to the membership date, the Infrastructure Team decided to have these emails sent outside of AMY. Maneesha wrote queries and scripts to be run weekly. This would identify all memberships that were entered in AMY in the past week and identify all memberships hitting their three month, six month, or renewal checkpoint in the past week. The script runs on a cron job. It creates a draft email in Helpscout to these members. This draft is reviewed by someone on the Membership Team (Angelique), who sends it if it looks good, or reports to Maneesha if there are any concerns. Planning work on this began in late 2021. System was implemented and moved to a monitoring phase in June 2022.

Next steps: This project is in a monitoring/maintenance phase now, and if all emails look good, we will move to having them all sent directly, rather than going through the draft review. These emails currently only cover standard memberships (Bronze, Silver, Gold, Platnimum). Future steps may also consider how we handle non-standard agreements.

maneesha commented 2 years ago

Instructor selection

Context: Currently when a new workshop request comes in, if it needs us to recruit Instructors, the Workshop Administrator manually enters the necessary information into a spreadsheet and sends a call out to our Instructors mailing list letting them know new opportunities have been posted. As Instructors sign up, the WA reviews signups, manually cross-checks them in the AMY database for things like teaching experience, areas of expertise, location, etc. Based on this the WA confirms instructors to teach. Through this project, we are moving signups to be inside AMY, making it easier for instructors and core team members to manage the process.

New System All instructor recruitment activity will happen within AMY. When a new workshop is entered in AMY, the workshop administrator will be able to open the instructor recruitment process for that workshop. A new view in AMY, available only to active certified Instructors, will allow them to see details of upcoming workshops and register their interest in teaching them. Still within AMY, the workshop administrator will be able to see who is interested in each workshop along with each instructor's profile, allowing them to easily confirm or decline the instructor to the workshop. This reduces administrative load on the Workshop Administrator and ensures none of the Instructor's information is publicly accessible. We are currently reviewing this system on the test AMY server with Core Team members and volunteer Instructors. Planning work on this project began in April 2021, with plans for implmentation in 2022 Q3.

Next steps Concurrent with release of this system will be release of a unified Carpentries Instructor badge. Currently, as a historical relic, we have three distinct instructor badges for the three lesson programs. There is no real distinction between the three badges though. This creates technological, administrative, and branding challenges. All current Instructors will be re-issued a Carpentries badge, and all new Instructors will get a Carpentries badge.

After the new system is implemented, we will also consider changes to automated emails (see above) and alerts when new teaching opportunities are posted.

maneesha commented 2 years ago

Open Training Invitations

Context: When new applicants were accepted through our open application program, a team member would periodically send them reminders to register for instructor training. This was done by manually querying accepted applicants in AMY, pasting emails in Helpscout, and sending mass email to all participants linking to our current calendar. Because Helpscout limits emails to 25 recipients, this often had to be done in multiple batches.

New system: Maneesha set up a cron job to query all individuals who have been accepted to instructor training but have not joined an event. It then sends each individual an email (rather than a mass bcc email) reminding them to register, and sends a log to the Instructor Training mailbox listing who has received this communication. This was set up in April 2021.

Next steps: No next steps are planned specifically for this automation, but other related steps may be considered in the context of general instructor training automations.

EDIT 2023-03-13 to add link to cron job.

maneesha commented 2 years ago

Instructor training planning for automations

Context Instructor Training administrative workflows are fragmented and time-consuming. Many of the most time-consuming steps could be automated. With Kelly stepping into another role in 2022 and many pressing needs for staff time elsewhere, improving efficiency with automation to the greatest extent possible is necessary and urgent. Several next steps have previously been identified, including import of attendance data to AMY. This project will additionally include a comprehensive review to ensure that requests for AMY development in particular are maximally efficient and effective, and that all impacts of change to existing workflows have been anticipated.

New system This was a meta-project. The Instructor Training team and Infrastructure Team reviewed all current workflows, and identified the most tedious, repetitive, or time consuming steps within each workflow. This allowed the Instructor Training team to note where changes to streamline processes could be made. We also identified what resources we would need to make these changes (Core Team time, external AMY developer time, or other resources). These discussions happened over 2022 Q2.

Next steps No next steps directly related to Instructor Training automations are being started, because this process highlighted that we need better planning systems around these type of work.

Out of these discussions, we developed a suggested plan to add requests like this (from all teams within the Core Team) to the Infrastructure Team road map. This will include:

  1. Conversations Maneesha has with each team about their needs
  2. Discussions within the Infrastructure Team about these needs
  3. Discussions at the Team Leads meeting to review these priorities and get them on the Infrastructure roadmap.
maneesha commented 2 years ago

Instructor training communications overhaul

Context The communications experience of a Trainee from first expressing interest in Instructor Training through certification and the associated Instructor Training Team workflows have developed organically over time. The process includes many touch points between trainees, Trainers, and The Carpentries Core Team. Frequent questions from Trainees suggested the process may be inundating Trainees with too much information in too many different places.

New System The team mapped out the course of communications that go out to trainees, and made changes to our email templates, website, handbook, and other workflows. By being methodical and intentional about all the information we were sharing, when it was being shared, we were able to move towards consistency and clarity in our communications. This sets us up well to scale our systems when we have opportunities to automate these workflows.

Next steps The Instructor Training team continues to monitor these new communication flows and identify areas for improvement. Many of these areas will be addressed in the next steps for Instructor training planning for automations.

maneesha commented 2 years ago

Instructor certificates in Helpscout

Context Instructor Certificates are generated and saved in a private GitHub repo. Approximately once a week, a Core Team member would send out certificates to all newly badged Instructor via a local script on their computer. There was no log of this, so we had no way to respond if a newly badged Instructor wanted to know if their certificate was sent out.

New System The script was updated so that it would not only send the email to each newly badged Instructor, but would also include additional inforamtion about joining The Carpentries community. It also sent a copy of the certificate to the Instructor Training mailbox, making it easier for Core Team to track when certificates were going out. This was implemented in January 2021.

Next steps While the system is improved, it is still manually run on a local Core Team member's computer. Ideally, this should be integrated into AMY so that an Instructor will get their certificate immediately after a badge is issued, and can access their certificate at any time by logging into AMY. Given AMY developer time is limited, this is likely low priority.

maneesha commented 2 years ago

Instructor training - checkout contribution directly in AMY

Context One of the three steps towards instructor training certification is a contribution to a Carpentries repo. This includes doing the following:

  1. Review the email
  2. Look up the profile in AMY
  3. Copy and paste the url to the contribution from email to AMY
  4. Mark it as passed
  5. Reply to the email.

New System AMY has always had the capability of letting trainees submit their contribution directly in AMY, but we have not publicized this. We are moving towards having them submit their contribution directly in AMY to reduce the administrative steps on our end (all we have to do is click "accept"). By having trainees log in to AMY directly before they get certified, this will also ensure they are ready to use our new system to sign up for workshops. This will reduce administrative burden on the Core Team from new Instructors who can't log in to AMY to sign up for upcoming workshops. A plan for this was developed in 2022 Q2, with implementation (next steps) in 2022 Q3.

Next steps A general plan for this has been developed. There are a few changes in AMY that will need to be made, and can be made concurrently with the work in the instructor selection project. At that time, we will also update our documentation and curriculum to reflect this change.

maneesha commented 2 years ago

Scheduling teaching demos and community discussions

Context Long ago, teaching demos and community discussions were scheduled at fixed times each week. This meant they did not account for the availability of Trainers and Discussion Hosts to run these sessions, and did not give community members enough variety of time slots to choose from.

New System In 2019, we began using Calendly so Trainers could choose their own slots to host teaching demos. In 2020, we included community discussion hosts to follow the same process. Each quarter, Trainers and Discussion Hosts are invited to use Calendly to sign up to run sessions at times that work best for them. Trainers'and Hosts' availability should be representative of our larger community, so the distribution of time slots should accommodate our global community.

In addition to using Calendly for Trainer/Host signups, we also use a script to generate the text for the Etherpad signup and a script to use Google Calendar's API to list these events on our community calendar. Both of these tasks had historiallly been done manually. We began this system in 2019 for Demo sessions and expanded to include Community Discussions for the 2022 Q3 sessions.

Next steps The Etherpad is not an ideal way for people to sign up. We have looked into other scheduling options but have not found ones that meet all our needs - including features like an API with GET and POST endpoints and ability for Trainers/Hosts to view registration and contact participants.

fmichonneau commented 2 years ago

Workshop survey results

Context To give access to the pre- and post-workshop survey results, we used to rely on SurveyMonkey featured. It required manual work as a link needed to be generated for each workshop based on the slug of the event.

New System Starting in late 2019, we switched to a new system where the surveys for the workshops are captured in Typeform. The data is then sent to Airtable. We developed a Shiny app that allows us to share data from the surveys with the hosts and instructors for a workshop. This system was extended in September 2021 to also include the results from the Instructor Training surveys.

Next steps Ideally, we would like to move away from Airtable and develop our own self-hosted database to capture the results from these surveys.

zkamvar commented 2 years ago

Lesson Website Automation: GitHub Actions

Context

From 2015 to the present, Carpentries lessons were built on top of a Jekyll template called carpentries/styles. The term "template" is a misnomer as it also contained the tools needed to build the lesson and these tools and stylings needed to be updated manually via pull request.

New System

The Carpentries Workbench is a new lesson infrastructure that separates content from tooling with R packages to build (sandpaper), validate (pegboard), and style (varnish) the lessons. We used GitHub Actions for deployment, with each repository containing a folder called .github/workflows/ that hold a series of workflows (which can be found at https://github.com/carpentries/sandpaper/tree/main/inst/workflows) that use GitHub Actions in https://github.com/carpentries/actions/ do deploy, validate, and maintain the workflows on a weekly basis.

These workflows are updated by creating a pull request on a weekly check when the workflows have updated. This process uses a token generated from the @carpentries-bot account (protected with 2fa) that is manually rotated on a schedule.

Because the maintenance workflows use a token, the github action check valid pr is used as a measure against malicious attacks via manipulation of the PR action.

Next steps

Lessons are to be deployed to an AWS S3 bucket folder using a unique key per lesson. Workflows components should be validated and updated with a defined schema so that authors outside of the carpentries can define their own steps should the need them and still benefit from the workflow updates using their own tokens. A secure system for token storage and regeneration may need to be considered.

maneesha commented 2 years ago

@kariljordan I saw you just assigned this to me. Is there anything else you need beyond what's shared in the comments above?

kariljordan commented 2 years ago

HI @maneesha, not at this time. I'm reassigning François' issues. Thank you so much.

ErinBecker commented 1 year ago

Adding @froggleston to this issue. Will discuss during his onboarding to the Core Team.

froggleston commented 1 year ago

Rob and @maneesha met today and we can provide this update:

Next steps for Q2/Q3

froggleston commented 11 months ago

Automated emails infrastructure

Instructor training planning for automations

Instructor certificates in Helpscout

Instructor training - checkout contribution directly in AMY

Scheduling teaching demos and community discussions

Lesson Website Automation: GitHub Actions

elletjies commented 6 days ago

Automation of Asana Template for Self-organised and Centrally-organised workshop administration