Closed cnk closed 3 years ago
zempo
commented
4 years ago I will make a test repo with mock data to try some solutions It will expand upon @cnk 's previous attempt to crawl through the project's directory.
rblaz001
commented
3 years ago I'm just getting over some pretty bad food poisoning so I haven't worked on this as much as I would have liked. I still need to make a 100 Automation's repo to continue developing my python script.
Action Item Progress
I have chosen to stick to a spreadsheet for the desired output of our script. I'm currently outputting a spreadsheet that contains two columns, one for the name of the project and the other for the list of missing data types.
The data types that we are using are currently hard coded in the script and were obtained by using a script that collected a set of used data types across all projects.
Set of Data Types
Project Missing Data Types
Note: I'm currently not taking account of nested data types. For example name, role, and links that are under "Leadership" in the following image. This is something I want to look in to, but I might have to reconsider how I deal with my output and determine what nested data types are important.
rblaz001
commented
3 years ago I now have a version of the script that outputs unused data types and missing data types for all projects. I still need create a 100 automation repo and add my work to it. Remaining work is quality of life edits to the script, including proper command line support.
The data types that we are using are currently hard coded in the script and were obtained by using a script that collected a set of used data types across all projects. The nested data types are denoted by comma delimited entries.
Set of Data Types
Project Unused Data Types
Project Missing Data Types (Currently Only One Missing)
Example of role missing from above entry
rblaz001
commented
3 years ago Added 100Automations repo. https://github.com/100Automations/script-gather-missing-data-types
alexandrastubbs
commented
3 years ago @ExperimentsInHonesty will talk to @rblaz001 about topic tags
ExperimentsInHonesty
commented
3 years ago @rblaz001 Please fill this out, so that we can get you listed on the 100automations.org site that will soon exist: create a project card for a 100 automations project
rblaz001
commented
3 years ago Spreadsheet for current unused/missing data types.
Note: Unused Data Types are data types that are never used in a project card. Missing Data Types are nested data types that are missing even though the parent data types are present. (This can be one or more instances)
rblaz001
commented
3 years ago Notes From Bonnie:
alexandrastubbs
commented
3 years ago @rblaz001 do you have a list of all the data types? If so, can you upload to google drive and share the link here?
rblaz001
commented
3 years ago @alexandrastubbs I have to make a small adjustment to the script to output all the currently used data types. I'm not home at the moment but I'll be able to upload the file and link it before tomorrow morning.
rblaz001
commented
3 years ago @alexandrastubbs I added an extra sheet labeled "Used Data Types" that has a list of data types used across the project cards.
https://docs.google.com/spreadsheets/d/1w59UfwbGiEAoBXjAxl4uwQe3xk5j6yFd3NxBMOm510Y/edit?usp=sharing
Note: The data types are not ordered. Also comma separated data types represent the nesting For example One,Two,Three would look like the following in the project card
One:
Two:
Three: valueProgress Report Progress - Was able to finish refactor, add JSON support, and update README. I was also able to update repo name and logo to include Jekyll Blockers - Need to show progress to PM and get notes Availability: 3-4 days depending on possible job interviews ETA:
Work In Progress Progress with automation card: https://github.com/100Automations/Website/issues/29
Finishing refactor documentation: https://github.com/100Automations/jekyll-gather-data-types/issues/2
Then working on validate through template feature: https://github.com/100Automations/jekyll-gather-data-types/issues/3
Was also able to look in to github action api and it should be possible to automate issues for missing data types relatively easy. The only blocker I'm having right now is thinking through how to manage issues so redundant issues are not created.
alexandrastubbs
commented
3 years ago @rblaz001 to rename from 'used data types' to 'all data types'
rblaz001
commented
3 years ago Progress - My availability changed this last week, I was unable to get any work done Blockers - None Availability: No more interviews or scheduling conflicts, will be available all week
Priority:
Task after priority finished:
ExperimentsInHonesty
commented
3 years ago @rblaz001 checking to see if you are able to put in an update on this issue before tomorrow.
rblaz001
commented
3 years ago Progress
Next step is to add validate through template.
Then begin work on implementing GitHub actions
Blockers - Moving to Austin, still trying to find an apartment so I have very limited time
Availability - Limited
ExperimentsInHonesty
commented
3 years ago Do UX requirement gathering. Bonnie will post on CfA slack asking for other people who have Jekyll sites and use collections.
rblaz001
commented
3 years ago template.md
---
alt: important
alt-hero: important
completed-contact: important
description: important
hide: ignore
identification: important
image: important
image-hero: important
leadership:
- links:
github: important
linkedin: important
slack: important
name: important
picture: important
role: important
links:
- name: important
url: important
location: important
looking:
- category: important
skill: important
partner: important
status: important
technologies: important
title: important
tools: important
---Updated Project Card Data Types - 9/13/2020 https://docs.google.com/spreadsheets/d/1F8ba2Kn6fjpp15zpseVqoYP7DsWVq1tpsKPusMCHF5Q/edit?usp=sharing
Updated Template Data Types - 9/13/2020 https://docs.google.com/spreadsheets/d/1KqqzZJOZKhX5QAB9X1wOwn_RuovRH1ikGwnQdUudbaM/edit?usp=sharing
rblaz001
commented
3 years ago Progress: Still have Validate by Template in Develop branch until I finish updating documentation. I hope to finish updating the README and pushing this to master today. Won't be able to make it to meeting today.
Blockers: Moving in a week
Availability: Today (Sunday 9/20) - I'm moving in a week, won't expect to work on anything until I'm settled in to my new place
rblaz001
commented
3 years ago Updated the README and merged Develop branch to master. I'll be driving to Austin starting Monday 9/28 and should be in my new apartment Thursday 10/1.
rblaz001
commented
3 years ago Progress Starting work on GitHub Actions for Jekyll Gather Data Types.
Still don't have much feedback on what a user expects for GitHub Action related features but will start by having issue generation based off of verify by template. Need to figure out a system that will track issues generated by Jekyll Gather Data Types to prevent redundant issue generation.
Blockers: Starting work and 10/12 and setting up a development environment on a new computer.
Availability: M-F 6:00 - 9:00 pm CT
rblaz001
commented
3 years ago No progress to report. Worked on wins page and previous status and availability still stands.
ExperimentsInHonesty
commented
3 years ago @rblaz001 checking in on you. Let's catch up on what you need from me on this issue.
ahdithebomb
commented
3 years ago @ExperimentsInHonesty messaged @rblaz001 the following via slack.
Hi Raudel hope everything is going well on your new job. Do you want us to hand this issue off to someone else, and if so, is there any information you need to impart before we do?
ExperimentsInHonesty
commented
3 years ago Review the feasibility of the following - Editing the script so that it produces the spreadsheet (instead of it being made manually) and another tab on the sheet that has only what project should have but is missing. In order to determine that, use the criteria below:
Current projects must have:
identification: ''
title:
description:
image: /assets/images/projects/[project repo name + .jpg or .png]
alt: 'description of the card image'
image-hero: /assets/images/projects/[project repo name + -hero + .jpg or .png]
leadership:
- name:
role:
links:
slack: 'https://hackforla.slack.com/team/xxxxxx'
github: 'https://github.com/githubusername'
picture: https://avatars.githubusercontent.com/githubusername
links:
- name: Github
url: 'https://www.example.com'
- name: Slack
url: 'https://www.example.com'
- name: Test Site
url: 'https://test.example.com'
- name: Demo Site
url: 'https://demo.example.com'
- name: Site
url: 'https://live.example.com'
- name: Overview
url: https://github.com/hackforla/product-management/blob/master/project-one-sheets/[REPLACE WITH PROJECT NAME]-Project-One-Sheet.pdf
# unused links can be commented out
# - name: Showcase deck
# url: ''
# alt: ''
looking:
- category: Development
skill: one skill
- category: UI/UX
skill: another skill
- category: Content
skill: Researcher
technologies:
- Node.js
- ReactJS
- Ruby on Rails
- other etc.
location:
- Remote
- Downtown LA
- Santa Monica
- South LA
partner:
tools: figma, photoshop, sketch, phone calls.
vertical:
status: ActiveCompleted projects must have
identification: ''
title:
description:
image: /assets/images/projects/[project repo name + .jpg or .png]
alt: 'description of the card image'
links:
- name: Github
url: 'https://www.example.com'
- name: Test Site
url: 'https://test.example.com'
- name: Demo Site
url: 'https://demo.example.com'
- name: Site
url: 'https://live.example.com'
- name: Overview
url: https://github.com/hackforla/product-management/blob/master/project-one-sheets/[REPLACE WITH PROJECT NAME]-Project-One-Sheet.pdf
# unused links can be commented out
# - name: Showcase deck
# url: ''
# alt: ''
technologies:
- Node.js
- ReactJS
- Ruby on Rails
- other etc.
partner:
vertical:
status: Completedthey can also have (but not mandatory)
image-hero: /assets/images/projects/[project repo name + -hero + .jpg or .png]
leadership:
- name:
role:
links:
slack: 'https://hackforla.slack.com/team/xxxxxx'
github: 'https://github.com/githubusername'
picture: https://avatars.githubusercontent.com/githubusernameOn Hold projects have everything that Active projects have except No Looking for Status: On Hold
ExperimentsInHonesty
commented
3 years ago @rblaz001 - sent you this on slack. Realized you might not check slack so sending it here too
Hi. Recently I made an update to your issue re the gather data types. I then unassigned you intending to assign a junior person to take a look at the feasibility. But I noticed today that the spreadsheet in the issue from the last time you produced a spreadsheet was now in the trash. Which makes me think you might have picked it up again.
So now I realize I should have started by asking you. Do you want some additional labor on this to produce the output (spreadsheet) we need to be able to use the scripts results on a weekly basis. I have a new backend dev that needs something meaty to work on.
If no, I will find something else for him
ExperimentsInHonesty
commented
3 years ago Also, I have suggested he look at the code you wrote for 100automations and see if he can do some of the things you asked for there (refactoring, etc).
ExperimentsInHonesty
commented
3 years ago sent raudel and email
akibrhast
commented
3 years ago Which fields should be allowed to be blank could you please specify using they keywords, REQUIRED and OPTIONAL like in the example below. @ExperimentsInHonesty @rblaz001
identification: ''
title: REQUIRED
description: OPTIONAL
image: /assets/images/projects/[project repo name + .jpg or .png]
alt: 'description of the card image'
image-hero: /assets/images/projects/[project repo name + -hero + .jpg or .png]
leadership:
- name: REQUIRED
role: REQUIRED
links: REQUIRED
slack: 'https://hackforla.slack.com/team/xxxxxx'
github: 'https://github.com/githubusername'
picture: https://avatars.githubusercontent.com/githubusername
links:
- name: Github
url: 'https://www.example.com'
- name: Slack
url: 'https://www.example.com'
- name: Test Site
url: 'https://test.example.com'
- name: Demo Site
url: 'https://demo.example.com'
- name: Site
url: 'https://live.example.com'
- name: Overview
url: https://github.com/hackforla/product-management/blob/master/project-one-sheets/[REPLACE WITH PROJECT NAME]-Project-One-Sheet.pdf
# unused links can be commented out
# - name: Showcase deck
# url: ''
# alt: ''
looking:
- category: Development
skill: one skill
- category: UI/UX
skill: another skill
- category: Content
skill: Researcher
technologies:
- Node.js
- ReactJS
- Ruby on Rails
- other etc.
location:
- Remote
- Downtown LA
- Santa Monica
- South LA
partner:
tools: figma, photoshop, sketch, phone calls.
vertical:
status: Active@rblaz001 @ExperimentsInHonesty I generated this following the requirements listed above. Is this the type of report you are expecting/wanting? Using the comment from @bonnie -> https://github.com/hackforla/website/issues/386#issuecomment-785427876 as the source of truth
cnk
commented
3 years ago What are the items like:
{
"filename": "food-oasis.md",
"errors": [
"is not of a type(s) string",
"is not of a type(s) array"
]
}ah.. my bad... let me work on a bit more detailed error report. But in answer to your question. The two errors in question are happening because these fields['looking','role'] in the image below are empt/null. Where my json schema is expecting them to be a string. @cnk
akibrhast
commented
3 years ago Edited https://github.com/hackforla/website/issues/386#issuecomment-790993825 , to give a bit more detailed report. I was really hoping to get something along the line of
"error":[
{
"message": "is not of type string",
"expected": "string",
"given": "null",
"stack": "on line 256 ,in 'message': null"
}
]
but can't seem to find the relevant information... :(
cnk
commented
3 years ago "looking" (and probably "technologies") can be empty. Can you express that in a json schema? optional fields can exist but be empty?
akibrhast
commented
3 years ago yewp, not a problem. I can make the looking be either string/null . But a project should always at least have technology, no?? I would be hesitant on allowing that to be empty...
akibrhast
commented
3 years ago Updated - > https://github.com/hackforla/website/issues/386#issuecomment-790993825 An empty looking field does not report an error. Although it is still a required key.
akibrhast
commented
3 years ago Currently I have a on: workflow dispatch action running that checks the files and generates a validation report. @ruben1s
Require feedback on where you want to go next with this @cnk? @ExperimentsInHonesty ?
This should be fairly easy to implement using this endpoint
https://api.github.com/repos/hackforla/website/issues?state=open&creator=akibrhast
Example Issue that would be created :
The following report was generated due to project schema not matching validation schema. Please review the report listed below under Project Validation Error Report and apply the necessary fixes.
ExperimentsInHonesty
commented
3 years ago @akibrhast I like the proposal.
Interested to hear your suggestions for improvement based on these concerns.
akibrhast
commented
3 years ago concerned about us determining that a project was not going to have something in it (like no partner) and no way to get it to stop flagging it.
Currently the schema is created based on this comment https://github.com/hackforla/website/issues/386#issuecomment-785427876 ('Current Project Must Have's') .
According to that comment, all current projects must have those keys. However it seems like if you look at the image below that key is commented out for some reason. https://raw.githubusercontent.com/hackforla/website/gh-pages/_projects/100-automations.md . That should not be the case, it's fine if the partner value is empty, but key should still exist.
https://hackforla.slack.com/team/xxxxxx .Same questions apply to Completed Projects and On Hold Projects.
Changing the schema is relatively simple. As you can see, a comment by @cnk https://github.com/hackforla/website/issues/386#issuecomment-791064664 was made and the schema was soon changed after that.
concerned that if it has one issue that has a lot of different projects in it, it will be hard to parse out. We can't get 10 PMs to weigh in on one issue.
Maybe generate a single issue per file ?
akibrhast
commented
3 years ago None since 5 days ago https://github.com/hackforla/website/issues/386#issuecomment-799935288
Awaiting Feedback from @ExperimentsInHonesty @cnk
Here and There
N/A
ExperimentsInHonesty
commented
3 years ago Met with @akibrhast - he will add notes to this doc.
Overview
We need an easy way of finding out when data is missing from project cards so that we can present an uniform UX and so that functions that rely on this data can work across all projects.
Action Items
Requirements