Get Dump of VRMS Data

[alexilee]

Overview

Get a dump of the VRMS database. Assess the data and report findings.

Action Items

• [x] Meet with Alex Anakin to understand data storage
• [x] Get a dump of the data
• [x] Parse and structure the data
• [x] Familiarize with the data and identify gaps
• [x] Run a brief analysis
• [x] Report findings

Resources/Instructions

VRMS Analysis Link VRMS Database Dump Link Roles

[alexilee]

Update: Met with Alex Anakin today. Got a high level introduction to the VRMS database. Alex will send me a dump of the data and be the point of contact with any questions I have.

Alex recommends to chat with Adam and Khushboo (VRMS PMs) regarding any data points we should be collecting that VRMS currently is not in order to get that included in the next VRMS release.

Additionally, Alex shared a link that contains the schema of Mongo DB, the VRMS production database. This gives the reader a list of data points in the database. Link below: VRMS Mongo DB Schema

[alexilee]

Alex Anakin has kindly provided me with a dump of the data from the VRMS database. He was kind enough to structure the data into CSV format. I have uploaded the CSV files to the Civic Structure GDrive: https://drive.google.com/drive/folders/1I7CCtGDUCShym_-qoVH922fl8JlSXQv7?usp=sharing

Only those with access to Civic Structure GDrive will have view access to the data since it contains PII.

[alexilee]

Question answered: What's the growth of HfLA membership? Analysis completed:

• Users created breakdown by month
• Users created breakdown by year

Question answered: What are the current roles of people joining HfLA? Analysis completed:

• All members current role breakdown
• Top 5 current roles

Question answered: What are the average number of checkins per project per event to get an estimate of team attendance? Analysis for Olivia completed:

• 2021 check ins by project

Need to do:

• Identify gaps in data fields to communicate to VRMS
• How many hours have volunteers spent?
• What's the percentage of technical vs non-technical volunteers?
• Other analysis requested

[alexilee]

Excel file attached with full analysis and references to figures. VRMS Dump.xlsx

Here is a summary of findings and recommendations to VRMS team.

Analysis Findings

1. No clear trends detected for how many volunteers are joining HfLA by time period. January and the first half of February of 2021 indicate a membership join rate significantly higher than 2020.
2. The top current role indicated by volunteers is an engineer/developer role accounting for more than 25% of all volunteers. See Figure 2A.
   • Volunteers categorized as underemployed consisting of unemployed, student, intern, self-employed and non-tech current roles make up 42% of all volunteers. See Figure 2B.
   • Volunteers categorized as tech make up 55% of volunteers. It is assumed that these numbers are inflated due to volunteers indicating a desired role as their current role and confusion in the sign up process. See Figure 2B. The top desired role indicated by volunteers is an engineer/developer role accounting for more than 40% of all volunteers followed by Designer and Product Managers at 18% and 10% of volunteers respectively.
3. Average checkins per event seem relatively low. It is likely that volunteers are not logging onto VRMS at a high rate. See Figure 3A. This analysis was requested by Olivia.
   • The data shows that there have been a total of 3,551 checkins and an estimated 7,102 hours volunteered assuming a two hour meeting per checkin. This is likely a low estimate due to sparse checkins from volunteers.

Notes to VRMS Team to Improve Data Collection

• The most important data we are missing is if and when users obtain a job. This is crucial to determining the success of the organization and we have no way to know this based on today's data.
• Users table does not have validation on creation and allows for duplicate users. Validation (on a field like email) should be implemented to prevent duplication of users.
• Users.currentRole and users.desiredRole fields are free text fields. They should be categorized into a dropdown menu for easier analysis.
• It seems that volunteers are more likely to designate that they have a specific current role even if they may be unemployed. There needs to be a clearer distinction in sign up between those currently employed and desired roles when asking volunteers for their current role.
• The Users table could contain other useful demographic information such as age, location, years of experience, currently employed, education level, technical vs non-technical etc.
• The Users table could also contain self assessment data to list skills or specific experience volunteers have such as data analysis or python.
• The Users table could also contain dropdown values of why users are joining HfLA whether they want to gain experience to find a job, improve communities through volunteering, etc.
• Users.newMember and Users.firstAttended fields o not seem useful and may be able to be deprecated.
• Checkins.checkedIn field does not seem useful and may be able to be deprecated.
• Events.checkInCount and Events.checkInReady fields do not seem useful.
• The Projects table does not seem to relate to other tables and does not seem useful in its current state. There should be more thought into what this table is to be used for.

[rfambro2]

@alexilee - This is amazing insight and really helpful for us to understand what data we currently have and what crucial information points are missing. I think this would be great to integrate into the Wiki somehow as we try to make it as comprehensive as possible. Maybe in the Output tab? However you see fit...this might even require a new tab.

[alexilee]

To Do:

• [x] Complete analysis of breakdown of desired role
• [x] Look at VRMS Wiki 'Getting Fully Up to Speed' and see which things are already on the roadmap versus things they might have missed
• [x] Compile a list of things we change right now for VRMS

Links: VRMS Wiki

[alexilee]

Notes to VRMS Team to Improve Data Collection Not Found in VRMS Future Release Requirements: Must-Haves

1. The most important data we are missing is if and when users obtain a job. This is crucial to determining the success of the organization and we have no way to know this based on today's data.

Should-Haves

1. High current role counts indicates that volunteers are more likely to designate that they have a specific current role even if they may be unemployed. It would be great to have a field that determines employment (like True/False)
2. Users.currentRole and users.desiredRole fields are free text fields. They should be categorized into a dropdown menu for easier analysis. The grouped fields used in this analysis can be a starting place for dropdown menu options. View excel file linked on this issue
3. The Users table could also contain dropdown values of why users are joining HfLA whether they want to gain experience to find a job, improve communities through volunteering, etc. This would help HfLA leaders collect data points to share with funders
4. Users table does not have validation on creation and allows for duplicate users. Validation (on a field like email) should be implemented to prevent duplication of users.

Could-Haves

1. The Users table could contain other useful demographic information such as age, location, years of experience, education level, technical vs non-technical etc.

Wish

1. The Users table could also contain self assessment data to list skills or specific experience volunteers have such as data analysis or python for ease of recruitment to projects.  

Taken Care of in VRMS Future Release Requirements:

1. Users.newMember and Users.firstAttended fields do not seem useful and may be able to be deprecated.
2. Checkins.checkedIn field does not seem useful and may be able to be deprecated.
3. Events.checkInCount and Events.checkInReady fields do not seem useful.
4. The Projects table does not seem to relate to other tables and does not seem useful in its current state. There should be more thought into what this table is to be used for.

[alexilee]

Here is the updated VRMS Analysis with charts and summary. VRMS Analysis Link

[alexilee]

Next steps:

• Use MoSCoW Prioritization to categorize recommendations for the VRMS team (edit comment above)
• Bring it to the team for review

[alexilee]

Next steps:

• [x] Reach out to VRMS PMS (Adam, Pete) and get agenda item added to discuss analysis and findings
• [x] Update Excel document to include MOSCOW prioritization
• [x] Convert Excel document into Google Sheet for ease of viewing
• [x] Start categorization and sub-categorization of roles. Use https://github.com/hackforla/civic-opportunity/tree/master/roles-1 as a starting place

[alexilee]

Next steps:

• [x] Send Bonnie a good job description for UX Team Lead position by Monday
• [x] Schedule a meeting with Aleiya Gafar to discuss her work on UX roles as seen on ticket Code for America Brigade Playbook
• [x] Check out article for competencies on UX: Article
• [ ] Research research operations role
• [x] Meet with Aleiya Gafar on UI/UX role findings
• [ ] Continue work on categorization and sub-categorization of roles: Roles

Resources: UI-UX Board Roles

[akibrhast]

Coming Back

[alexilee]

Met with Aleiya Gafar (ag2463 on codeforamerica repo) to discuss her findings on UI/UX roles. She was tasked to create job descriptions for Research Ops Coordinator, UX Research Lead and UX Program Designer roles. She didn't find too many roles out there for Research Ops Coordinator, maybe this isn't a role we should consider incorporating into the org. We decided she will hold off on completing this job description, and the UX Program Designer position until I finalize which positions we will have.

UI/UX Role Descriptions

[alexilee]

Conversations from Slack with Aleiya, Danielle, Bonnie, Marianne, Serena, Olivia and Alex:

From Bonnie Hi all the people with a specific interest in recruiting someone to replace the coordination of UX researchers on the BOP project (currently being handled by Mari and Serena). Hack for LA has a vested interest in helping this role get filled. So please understand that when I am pushing you all to coordinate and come up with appropriate titles for the UX roles, it is because only that will enable us to recruit people for this role. Any recruiting for this role without first understanding the market that people are applying into is not in keeping with Hack for LA’s values and will not result in a pipeline of people being available. Today I received a message from Serena "Hey Bonnie How are you? I am hoping to recruit research ops and a data analyst today which channel should I join for this?" I have asked Both @Aleiya Gafar and @Alex Lee to work on sorting out the UX roles that we recruit for at Hack for LA, what they are called and what their responsibilities are. Aleiya is the PM responsible for getting this position filled on the BOP team. Alex is responsible for making sure that the roles fit into a comprehensive set of roles that are available to all Hack for LA members and ultimately that those roles have the same titles and map to roles available on places like Linkedin and indeed. Here are the roles as I understand them both for other Hack for LA projects as well as BOP UX Researcher (suitable to someone with little to no training or experience. We will train them to do the interviews, transcription validation, usability surveys, etc.):

• [AS YET UNTITLED] A person to coordinate the work activities of all UXRs (suitable for someone who has the time coordinate all the UXRs and is open to being trained to do all the necessary functions. And who would like to get managing a team on their resume.
• Instructional Designer A person to write the training on how to do the UX activity. Currently being filled by David Russo on BOP
• [AS YET UNTITLED] A person who writes up a comprehensive research plan for a component of the research (survey, usability tests, UX workshops, etc.).
• [AS YET UNTITLED] A person who writes a comprehensive roadmap for all the research, analysis and presentation to be completed on a particular project.

From Alex I’ve laid out three positions here that are very common in the corporate world and their potential application to Hack for LA:

• UX Research Coordinator - the most entry level position. This person is responsible for scheduling research, the recruitment of study participants and communication with study participants including tool administration and incentive management. This role is more relevant in large UX departments so I doubt HfLA needs this role as these responsibilities can be handled by the UX Researcher.
• UX Researcher - entry level to senior. This person is responsible for identifying research opportunities, planning and designing studies and executing on and synthesizing results of said studies. This role is most relevant to HfLA volunteers and the majority of folks should be in this role.
• Lead UX Researcher/UX Research Lead - senior. The individual contributor job responsibilities of this role are identical to the UX Researcher role. In addition, this person is also responsible for training new members, setting best practices and guides and collaboration with other stakeholder leads. This person should be the one with some management responsibilities. Most projects at HfLA will not need this role, it should be reserved for those who are focusing on training and development of new members and communication across projects.

I recommend against having a UX Research Coordinator position at HfLA due to our size and maturity. I also would recommend against a Research Ops role as this is still an emerging discipline and so it may be hard for volunteers to find a job with this title afterwards. Additionally, the Research Ops role makes more sense after you have your basic UX Research practices set up so that you can focus on scaling, which is not where HflA is. Instead of these two role, I recommend the role below:

• UX Program Manager - mid-level. Most small companies do not have a need for this role. This person is responsible for managing larger design programs (for both design and research) that span across projects, helping coordinate the UX roadmap against multiple projects and communication between other program and project managers. While HfLA’s design team isn’t mature yet, it may make sense to have this role to assist in process creation and coordinating across communities of practice.

I’ve mapped Bonnie’s 5 points from her message above to the following roles:

1. UX Researcher
2. Coordinating across all UXR = UX Program Manager. Managing responsibilities = UX Research Lead
3. Writing trainings or guides = UX Research Lead or UX Program Manager
4. Creating a research plan for a component = UX Researcher
5. Roadmap for all the research pertaining to a project = UX Researcher or UX Research Lead (depending on if there is a lead on the project) These are my suggestions. Happy to hear everyone’s thoughts here.

[alexilee]

Alex to create a document or spreadsheet with this information for ease of sharing and reading.

[alexilee]

Ticket needs review from Bonnie after her move. Last we spoke, she needed to think about the presented roles for UX and get back based on the needs.