search

hackforla / website

Hack for LA's website
https://www.hackforla.org
GNU General Public License v2.0
295 stars 713 forks source link

Update wiki page for GitHub Action: github-data #6643

Closed roslynwythe closed 3 weeks ago

roslynwythe commented 2 months ago

Overview

We need to review and update the wiki page for the GitHub Action github-data. In particular we need to create two new sections that have recently been added to the GHA wiki page template: "Labels" and "Test Procedure".

Action Items

Merge Team/Dev Lead

Resources/Instructions

github-actions[bot] commented 2 months ago

Hi @roslynwythe.

Please don't forget to add the proper labels to this issue. Currently, the labels for the following are missing:

NOTE: Please ignore this comment if you do not have 'write' access to this directory.

To add a label, take a look at Github's documentation here.

Also, don't forget to remove the "missing labels" afterwards. To remove a label, the process is similar to adding a label, but you select a currently added label to remove it.

After the proper labels are added, the merge team will review the issue and add a "Ready for Prioritization" label once it is ready for prioritization.

Additional Resources:

ExperimentsInHonesty commented 2 months ago

@roslynwythe

ExperimentsInHonesty commented 2 months ago

Also, I don't really understand what the value of the inclusion of this is

I read it, but it was not clear what that was in reference to. We generally don't want to link to a comment that then requires us to scroll up to prior comments to understand what it's related to, because that can lead someone down a rabbit hole. If we could summarize that or give context before they view the link, that would work.

roslynwythe commented 1 month ago

@roslynwythe

Please see my response https://github.com/hackforla/website/issues/6403#issuecomment-2067931770

roslynwythe commented 1 month ago

Also, I don't really understand what the value of the inclusion of this is

I read it, but it was not clear what that was in reference to. We generally don't want to link to a comment that then requires us to scroll up to prior comments to understand what it's related to, because that can lead someone down a rabbit hole. If we could summarize that or give context before they view the link, that would work.

You are corret, it is not relevant. I had copied the Resources/Instructions section from a different issue and included that one by mistake.

github-actions[bot] commented 1 month ago

Hi @YolandaHaynes, thank you for taking up this issue! Hfla appreciates you :)

Do let fellow developers know about your:- i. Availability: (When are you available to work on the issue/answer questions other programmers might have about your issue?) ii. ETA: (When do you expect this issue to be completed?)

You're awesome!

P.S. - You may not take up another issue until this issue gets merged (or closed). Thanks again :)

YolandaHaynes commented 1 month ago

Availability: Wed - Thur 8:30-10:30pm, Sun 1-4pm ETA: EOD 4/28/24

YolandaHaynes commented 1 month ago

Udpating my timeframe due to time conflicts last week. Roadblocks: trying to understand the GHA and not finding/understanding what 'labels' to add for the new section. Continuing reading and understanding the concepts. Availability: Tue/Thur 8:30-11:30pm pst ETA: 05/03/24

roslynwythe commented 1 month ago

Hi @YolandaHaynes It appears that there aren't any labels relevant to this GHA, so you can just write "none".

YolandaHaynes commented 1 month ago

Summary

The schedule-daily-1100.yml GitHub Action retrieves Hack for LA's project repository information using the get-project-data.js script. It then overwrites the github-data.json file with the current repository data and commits it to the website repository. The new data will then display in each project's page. This action is triggered daily by a cron job.

Scheduled

Labels

Workflow

  1. Retrieve old github-data.json
  2. Retrieve all tagged and untagged repos
  3. For each repo:
    1. Retrieve:
      • id
      • name
      • languages
      • url
      • Issue Contributors
      • Issue Comment Contributors
      • Project Contributors
    2. Append data in format below: 
      {
id: repo.id,
name: repo.name,
languages: Object.keys(repoLanguages.data),
repoEndpoint: repo.url,
commitContributors: {
  data: commitContributors
},
issueComments: {
  data: issueCommentContributors
},
contributorsComplete: {
  data: projectContributors
},
}

Supporting Files/Folders

Test Procedure

  1. In order to start testing, you will need to create the HACKFORLA_PA_BOT_TOKEN in your personal repo. Review the GitHub documentation for creating a personal access token . When creating the token, select the following scopes:

    • public_repo (under 'repo')
    • admin:org_hook

  2. Then you can add the token to your personal repo using these instructions.

  3. Update the schedule-daily-1100.yml workflow file by adding the workflow_dispatch command below. This will allow you to trigger the GHA manually on your working branch so that you can test it.

  4. Change the github.repository from hackforla/website to your-github-handle/your-repo-name.

on:
  schedule:
    - cron:  '0 11 * * *'
  workflow_dispatch:

jobs:
  github_data:
    runs-on: ubuntu-latest
    if: github.repository == 'your-github-handle/your-repo-name' # Usually the repo name is `website` unless you changed it while forking the main Hack for LA repo

  1. Change your default repo to your working branch using these instructions.

  2. Since this GitHub Action updates data about Hack for LA's many projects/repos, you will need to replicate the Hack for LA organization to test this workflow. a. Create a new organization. GitHub official docs b. Transfer your repo to the newly created organization. GitHub Doc on how to transfer c. Create a couple of test project repositories with contributors, code, url, language etc as if it's a real project repo. An easy way to do this may be to fork some of Hack for LA's existing projects.

  3. After you have created a mock scenario, trigger the GHA by pushing to your repo.

  4. Make sure to test both happy and unhappy paths and consider possible edge cases.

  5. The expected result is that the workflow will overwrite the existing github-data.json file with the new data. You should see the data you created reflected there. Given your personal repo may not possess all of the projects that exist on Hack for LA, keep that in mind as you test. Check to see if the new changes made it to the github-data.json file.

[!CAUTION] Before you submit a PR, do the following:

  1. Remove the workflow_dispatch command
  2. Change github.repository back to hackforla/website
  3. Change your default repo back to gh-pages
  4. Revert any changes you made to the github-data.json file for testing on your branch

Still work in progress

Questions:

  1. Am on the right path with the testing procedure instructions for the GHA?

  2. I'm not entirely sure if I am understanding this GHA. I added my summary at the top from what I gathered reading through the resources. So when this GHA gets triggered, it updates the github-data.json file with new data. Does that mean when this json file gets updated, then in the Hack for LA's project section, each Project's page will have the most up-to-date information? Is this the reason behind this GHA?

  3. How is the data in github-data.json used? Will it automatically display on the projects page?

YolandaHaynes commented 1 month ago

Hi @roslynwythe , I'm unsure about the testing procedure for this GHA. Please let me know if I'm on the right path with what I have written above.

t-will-gillis commented 1 month ago

Hey @YolandaHaynes Thanks for working on this! I have some comments:

Screenshot 2024-05-16 122627

t-will-gillis commented 1 month ago

PS I would recommend adding the quotes around 'hackforla/website' instead of just hackforla/website - I am saying this because if the quotes aren't included, the workflow will crash.

github-actions[bot] commented 1 month ago

@YolandaHaynes

Please add update using the below template (even if you have a pull request). Afterwards, remove the 'To Update !' label and add the 'Status: Updated' label.

  1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
  2. Blockers: "Difficulties or errors encountered."
  3. Availability: "How much time will you have this week to work on this issue?"
  4. ETA: "When do you expect this issue to be completed?"
  5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the Questions/In Review column of the Project Board and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel. Please note that including your questions in the issue comments- along with screenshots, if applicable- will help us to help you. Here and here are examples of well-formed questions.

You are receiving this comment because your last comment was before Tuesday, May 14, 2024 at 12:06 AM PST.

YolandaHaynes commented 1 month ago
  1. Progress: "Received some feedback and will finish up the task"
  2. Blockers: None at the moment.
  3. Availability: Sunday 11-1pm, Tues/Thur 9-11pm
  4. ETA: If no other roadblock surfaces, EOD Tue
YolandaHaynes commented 4 weeks ago

Hi @t-will-gillis Thank you so much for your feedback. Below I have updated with your input. Please let me know if I misunderstood something or missed anything else.

For the workflow section you made the comment about, I just took the information from https://github.com/hackforla/website/wiki/GHA:-github-data. I thought that information was already updated and verified since it was last edited in 2023. I slightly updated the workflow section from your input.

Updated version below:

Summary

The schedule-daily-1100.yml GitHub Action retrieves Hack for LA's project repository information using the get-project-data.js script. It then overwrites the github-data.json file with the current repository data and commits it to the website repository. The new data will then display in each project's page. This action is triggered daily by a cron job.

Scheduled

Labels

Workflow

  1. Retrieves each project in HFLA contributions github-data.json
  2. Retrieve all tagged and untagged repos
  3. For each repo:
    1. Retrieve:
      • id
      • name
      • languages
      • url
      • Issue Contributors
      • Issue Comment Contributors
      • Project Contributors
    2. Append data in format below: 
      {
id: repo.id,
name: repo.name,
languages: Object.keys(repoLanguages.data),
repoEndpoint: repo.url,
commitContributors: {
  data: commitContributors
},
issueComments: {
  data: issueCommentContributors
},
contributorsComplete: {
  data: projectContributors
},
}

Supporting Files/Folders

Test Procedure

  1. Change your default repo to your working branch using these instructions.

  2. To set up your testing environment, please follow the steps here: Set-Up Environment

  3. In order to start testing, your GITHUB_TOKEN is automatically added, and you don't need to add it explicitly. You can read the GitHub official doc under secretes in workflow. You will need to create the HACKFORLA_PA_BOT_TOKEN in your personal repo. Review the GitHub documentation for creating a personal access token . When creating the token, select the following scopes:

    • public_repo (under 'repo')
    • admin:org_hook

  4. Then you can add the token to your personal repo using these instructions.

  5. Update the schedule-daily-1100.yml workflow file by adding the workflow_dispatch command below. This will allow you to trigger the GHA manually on your working branch so that you can test it.

  6. Change the github.repository from 'hackforla/website' to 'your-github-handle/your-repo-name'. Also remember to update your token from HACKFORLA_PA_BOT_TOKEN to your newly created token ${{ secrets.Your_New_Token }}

on:
  schedule:
    - cron:  '0 11 * * *'
  workflow_dispatch:

jobs:
  github_data:
    runs-on: ubuntu-latest
    if: github.repository == 'your-github-handle/your-repo-name' # Usually the repo name is `website` unless you changed it while forking the main Hack for LA repo

  1. To trigger the GHA, go to the "Actions" tab, select "Schedule Daily 1100" on the left, and then selecting "Run workflow" button to the right.

  2. A successful test will most likely update the data in github-data.json, but this is not guaranteed. The action stefanzweifel/git-auto-commit-action@v5.0.1 checks the existing data file and only updates when and if it detects changes. If you were to run simultaneous tests of "Schedule Daily 1100", there likely would be no changes to the data, so nothing new would be written, but this is ok. The successful test is to check whether the test run is 'all green' and whether the auto-commit step either determined that the data was already up-to-date (thus did not update anything) or the auto-commit updated the data.

[!CAUTION] Before you submit a PR, do the following:

  1. Change github.repository back to hackforla/website
  2. Change the token back to ${{ secrets.HACKFORLA_BOT_PA_TOKEN }}
  3. Change your default repo back to gh-pages

Thank you again @t-will-gillis for your guidance. I wasn't able to test before because I wasn't sure if I was on the right path with how to test this GHA initially. I am now able to test it on my end and I had a successful test with a green check. Please let me know if some of the instructions are not clear enough, or I put in too much information.

t-will-gillis commented 3 weeks ago

Hi @YolandaHaynes - The instructions before you worked on this were pretty sparse, so this is a great improvement! It is looking very good and I think you are giving the right amount of information.

Here are a couple more suggestions, if you agree. (I did not copy the links to the snippets below). For the Summary, this might be clearer:

The [schedule-daily-1100.yml]() GitHub Action retrieves Hack for LA's team member contributions from each project’s repository using the [get-project-data.js]() script. It then updates the [github-data.json]() file …

And for the Workflow:

  1. Retrieves member contribution data from each HFLA project's repo and saves it to [github-data.json]()

Thanks again for working on this!

YolandaHaynes commented 3 weeks ago

Summary

The schedule-daily-1100.yml GitHub Action retrieves Hack for LA’s team member contribution from each project’s repository using the project get-project-data.js script. It then updates the github-data.json file with the current repository data and commits it to the website repository. The new data will then display in each project's page. This action is triggered daily by a cron job.

Scheduled

Labels

Workflow

  1. Retrieves member contribution data from each HFLA project’s repo and saves it to github-data.json
  2. Retrieve all tagged and untagged repos
  3. For each repo:
    1. Retrieve:
      • id
      • name
      • languages
      • url
      • Issue Contributors
      • Issue Comment Contributors
      • Project Contributors
    2. Append data in format below: 
      {
id: repo.id,
name: repo.name,
languages: Object.keys(repoLanguages.data),
repoEndpoint: repo.url,
commitContributors: {
  data: commitContributors
},
issueComments: {
  data: issueCommentContributors
},
contributorsComplete: {
  data: projectContributors
},
}

Supporting Files/Folders

Test Procedure

  1. Change your default repo to your working branch using these instructions.

  2. To set up your testing environment, please follow the steps here: Set-Up Environment

  3. In order to start testing, your GITHUB_TOKEN is automatically added, and you don't need to add it explicitly. You can read the GitHub official doc under secretes in workflow. You will need to create the HACKFORLA_PA_BOT_TOKEN in your personal repo. Review the GitHub documentation for creating a personal access token . When creating the token, select the following scopes:

    • public_repo (under 'repo')
    • admin:org_hook

  4. Then you can add the token to your personal repo using these instructions.

  5. Update the schedule-daily-1100.yml workflow file by adding the workflow_dispatch command below. This will allow you to trigger the GHA manually on your working branch so that you can test it.

  6. Change the github.repository from 'hackforla/website' to 'your-github-handle/your-repo-name'. Also remember to update your token from HACKFORLA_PA_BOT_TOKEN to your newly created token ${{ secrets.Your_New_Token }}

on:
  schedule:
    - cron:  '0 11 * * *'
  workflow_dispatch:

jobs:
  github_data:
    runs-on: ubuntu-latest
    if: github.repository == 'your-github-handle/your-repo-name' # Usually the repo name is `website` unless you changed it while forking the main Hack for LA repo

  1. To trigger the GHA, go to the "Actions" tab, select "Schedule Daily 1100" on the left, and then selecting "Run workflow" button to the right.

  2. A successful test will most likely update the data in github-data.json, but this is not guaranteed. The action stefanzweifel/git-auto-commit-action@v5.0.1 checks the existing data file and only updates when and if it detects changes. If you were to run simultaneous tests of "Schedule Daily 1100", there likely would be no changes to the data, so nothing new would be written, but this is ok. The successful test is to check whether the test run is 'all green' and whether the auto-commit step either determined that the data was already up-to-date (thus did not update anything) or the auto-commit updated the data.

[!CAUTION] Before you submit a PR, do the following:

  1. Change github.repository back to hackforla/website
  2. Change the token back to ${{ secrets.HACKFORLA_BOT_PA_TOKEN }}
  3. Change your default repo back to gh-pages
YolandaHaynes commented 3 weeks ago

Attached above comment into the How to Contribute Wiki Page under the GHA: github data section.

@t-will-gillis I’m a little bit confused as to how to have this peer reviewed. Please let me know what steps I need to do next to start that process. Thank you.

t-will-gillis commented 3 weeks ago

Hey @YolandaHaynes - Looks great and thanks so much for all of your work! (And thanks for putting up with the review.) In retrospect, I think this was much more than a Complexity: Medium...

You added the link to the Wiki, and what we've been doing counts as peer review, so I will 'Close as Completed' and you can move on to your next issue.