help wanted label (mini project)

[nelsonic]

As a member of the dwyl team/community I would like to have a dashboard of all issues that have the help wanted label applied and are not assigned to someone already so that I can see where people need help.

GitHub has the basic functionality to search for label across the org: https://github.com/search?q=org%3Adwyl+is%3Aissue+is%3Aopen+label%3A%22help+wanted%22 and even filter by a criteria:

But even the "advanced search" does not have the ability to filter the issues to see which ones have been "estimated" or which are assigned/unassigned so people know who has already agreed to collaborate on solving a challenge... we need the ability to use issues for effective communication in our teams/community.

We need:

• [ ] a script or micro-service that uses OAuth to fetch all the issues in the Organisation see: https://github.com/dwyl/github-backup/issues/1 (this script is it's self a "mini-project")
• [ ] a functional list view where all help wanted issues can be seen.
• [ ] clearly show the issues which have/not been assigned to anyone (i.e. still needing help!)
• [ ] clear visibility when a person has assigned the issue to themself (i.e. the issue is already being worked on by a person/team so should not appear on the list of issues still requiring help)
• [ ] documentation
• [ ] tests

for inspiration of the base code required to build this see: https://github.com/dwyl/labels

If you are interested in taking on this mini-project as your first bit of #RemoteWork please leave a comment so we can discuss further!

Meanwhile if anyone else is interested in helping us check out: https://github.com/search?utf8=%E2%9C%93&q=org%3Adwyl+label%3Apriority-1+label%3A%22help+wanted%22+is%3Aopen+&type=Issues&ref=searchresults

[Jbarget]

@nelsonic @des-des and I also have some boiler-plate code for this

https://github.com/Jbarget/oauth-example/blob/master/lib/server.js

As it stands:

• oAuth Github Login
• bare access_token being stored as a cookie
• making authenticated request for /user
• partially tested

Do you have any ideas re wireframes for MVP?

[nelsonic]

Hi Justin! Really sorry I didn't reply when I first read this... the perils of getting distracted by family when you are reading GitHub notifications... 😩 Yes, kinda do have an idea, essentially we need to display the data in a table with sortable columns.

[Jbarget]

no worries at all!

accoriding to your screenshot above:

I think that thats not possible call to the github API ie. theres no option to list out all issues within an organisation. You can for a repository though.

So it might need to be something like:

• Choose an organisation
• Choose a repo ---> lists out all issues with 'help wanted' label in that repo

[nelsonic]

Yeah, I suspect you are correct. (can't call GitHub REST API with a specific query for a label...) But this mini-project is part of a larger project to extract all issues across all repositories. Then we can filter the help wanted and display them in a useful format. Ultimately our objective is to visualise all the help wanted across the organisation so that when people ask us: how they can help us make "progress": https://github.com/dwyl/hq/issues/165) we can just point them to the list and they can pick off something to work on... 👍

The other benefit_ to "mirroring" all the issues in the organisation is that we can still get work done when GitHub has the next DDOS attack... https://github.com/dwyl/github-reference/issues/15

[SimonLab]

So to recap and to check I understand the structure of this project: this project can be split into two smaller projects:

• A backup system where all the issues of an organisation are saved in our own database (lambda? do we need a UI?)
• An API + UI to query directly our saved issues (instead of using the Github API) and see specific issues ("help-wanted" and other filters)

@nelsonic is this right?

I've started to play with Elm and I'm currently trying to extract all the "help-wanted" issues from a list of repositories. At the moment it's just a simple SPA to just help me to learn how to create http requests from Elm and how to parse Json to Elm types. However I'm happy to have a go at this project and try to create a better structure with Elixir (Phoenix?) and Elm. We might need first to define the requirements in details on how to backup Github issues. @Jbarget have you done some progress on this?

[nelsonic]

Hi @SimonLab & @Jbarget, Yes, there are two distinct "parts" to this work. The "Architecture" for the "app" is quite simple:

We need to "white board" the process. (when do you have time on Wednesday?) I would like to "deduce" the Schema for each PostgreSQL table based on the GitHub API rather than manually write out each table/field.

[SimonLab]

Wednesday afternoon is good for me, @Danwhy are you interested too?

[Jbarget]

@SimonLab nope i havent made any headway with this

[jackcarlisle]

I'd be keen to work on this!

[nelsonic]

Sweet Y'all! 🍭 (great to hear there are smart people interested in making DWYL - and other orgs that use GitHub for issue/project tracking - a well-oiled-machine!!)

We can significantly speed up our "workflow" and teamwork with this. Just by:

• having a faster caching
• hitting the GitHub API after the page loads
• updating the content in response to any new content returned by the API

we can shave off seconds per action/task (which add up to hours over a year!) and most importantly integrate "real-time" into our workflow without needing "Gitter" (or other "chat" app to inform each other that there is a new question/issue/bug/PR ...)

There many useful features we can build using WebSockets ... so brush up on your skills if they are "rusty".

Please focus on getting up-to-speed with Elixir & Phoenix as we aren't going to be building "new" things in JS/Node anymore... 😮 relates to: https://github.com/dwyl/technology-stack/issues/37 🚀

[samhstn]

I can see this project already has received enough interest from some genii, but if there's a space going, I'd love to help out.

[jay-meister]

I am with @shouston3 on that. Sounds like a fun project!

[iteles]

Lots of interest, but no one has claimed it yet...

[jackcarlisle]

@iteles All jokes aside, it there a process for claiming a dwyl project? Or is a selection made based on the people who have shown interest? Or should we just begin working on it?

[des-des]

Yeah I'm down

[Jbarget]

@des-des and I are keen to take this on while we're in Nazareth. Eoin and I are both without projects at the moment and would be keen to take this on, would you be ok with that?

We have an idea of next steps but want to wait for approval :)

[nelsonic]

@Jbarget & @des-des great! 🎉 how comfortable are you with "PETE" Stack? 🤔 https://github.com/dwyl/technology-stack#the-pete-stack

[Jbarget]

@des-des has some Elm in his pocket, I've touched on it. In terms of Elixir/Pheonix I've been going through Udemy's elixir/pheonix course and seems like there are specific sections of that which link directly to the architecture above.

We were thinking about doing a day of unpaid research into the stack before starting (could be today/tomorrow?)

[jackcarlisle]

@Jbarget I'd recommend this book for getting started with Phoenix if you can get your hands on one.

A couple of copies have been floating around Quiet Space and it's what I've found to be the most useful. I've also done that Udemy course but it doesn't cover a lot of things that the book does in detail.

[Jbarget]

@jackcarlisle ace, although i imagine the postal service to Nazareth might be a bit unreliable :)

[jackcarlisle]

@Jbarget yeah that's what I thought!

[Jbarget]

@nelsonic we're thinking next steps are:

• day of research into PETE stack
• agree scope for project
• time estimate for scope

[nelsonic]

Jbarget learning "PETE" stack might take more than a day ... 🤔 (although @jruts managed to learn from scratch, deploy his first app and PR an learning update in less than that ... but he's an exception to most rules!) 😉

See how you get on and report progress! If you can help us improve any of the dwyl intro tutorials it would be amaze! our goal for the year is to help everyone transition to "PETE". 🚀

[Jbarget]

sounds good, @des-des and I we're thinking about spending the rest of the day/tomorrow reading up on the PETE stack and scoping this. Before that we wanted to confirm that if scoping goes well, then you would be happy for us to move forward within the next few days.

Its fairly pressing as we're expecting both of our workloads to increase soon and would be good to make a start before then.

[des-des]

@nelsonic @iteles yeah so me and @Jbarget have been been working through programming phoenix / Programming Elixir for the last 5 days. We are feeling confident we can take this on, but we are not sure what steps we need to take to claim this project...

If you want someone to build this we are happy to do it and will do a good job.

We have 2 questions:

1. Do you have a budget for this project?
2. What do we need to do to claim this project?

[iteles]

@des-des @Jbarget The only steps required are:

• Time estimate for the project (based on @nelsonic's checklist in the top comment in this issue)
  • Bonus level: Customisation to allow user to input any organisation
• Get approval from @nelson or myself to go ahead (a conversation on rates will need to be had)
• Build!

Remember that for a first pass, it just needs to be functional so that we can validate the need as an MVP. Once that is done and feature requests start pouring in, we'll ask @harrygfox to take a look at the UI/UX and go from there.

To note: All time estimates should include the usual documentation and tests and projects will need to follow our contributing process.

[des-des]

Time estimates

These estimates are for two people (so double them for work days)

We need:

• [ ] a script or micro-service that uses OAuth to fetch all the issues in the Organisation
  • [ ] oauth 1 day
  • [ ] data fetching (build urls + manage pagination) ~2 day~ 1.5 day
  • [ ] Build Ecto models to represent gh data + migrations ~2 days~ 1 day
• [ ] Functional list views where help wanted issues can be seen.
  • [ ] default view: all help wanted issues (assigned and unassigned). 1 day
  • [ ] unassigned view: all help wanted issues with no assignee .5 day
  • [ ] assigned view: all help wanted issues with an assignee .5 day

Thoughts:

Here is our first guess in terms of what could be done with caching functionality, but am sure there are lots of different approaches here..

There are two first steps here:

1. For each request we could re-fetch all the issues from gh, and keep them in postgresql as a fallback for when the github api is down.
2. On request do 2 things simultaneously :
   1. Serve pages immediately from db / cache,
   2. Make request for data from gh and update db. Here we could use since param query on gh request to only get issues that have changed

We think the second option here is much more desirable. Although the data may be a bit stale, we can make the page load fast and add functionality to keep the db fresh without change the structure of our application too much.

Ordering We are assuming we will order them by last updated (newest first). Pagination How many issues do we display per page? (we will make this stretch goal)

[nelsonic]

@des-des thanks very much for adding detail on your thoughts & time estimates.

Are your estimates for a single person or pair? 🤔 as @iteles says please agree rates and ensure your "freelancer agreement" is up-to-date before starting.

@SimonLab / @des-des / @jackcarlisle / @shouston3 / @Jbarget / @JMurphyWeb I've split the first part of this mini-project into a separate script, please see: https://github.com/dwyl/github-backup/issues/1

We cannot visualise the help wanted issues for a given Organisation without first retrieving the data from GitHub. I tried running: https://hackage.haskell.org/package/github-backup but could not get it to compile... Who ever is not currently working on a client project can take a look at this.

Having a project called github-backup that works and has some actual documentation will be good.

[des-des]

@nelsonic Yeah, these estimates are for a pair (pairing). Will talk to @iteles!

We think that for this project a db would be better than using the filesystem. Crawling the file system to find the most recent issues will add unnecessary complexity + ugliness to the application (ie I think your initial spec would work better than building on top of gh-backup!).

Maybe once we create this, we can reuse / abstract the gh oauth / requests so that someone taking on this new project can reuse them!

[nelsonic]

@des-des sorry for the confusion, I wasn't suggesting that we should crawl FS for tudo. I was suggesting that as an independently useful tool github-backup should save records to FS for easier access for people who don't want to automatically load their GitHub issues into a DB.

For our purposes we could just fetch the records from GitHub REST API and pipe them into PSQL.

So why the "extra step"...? To break it down into measurable chunks that can be re-used by other people. The more small (incredibly) useful things we make the more value we give back to the community. Which in turn attracts more people to contribute to dwyl's mission. ❤️ ✅ if we leave the "make it immediately useful to other people" step till last it won't get done.

This is the philosophy behind writing tutorials/examples e.g: https://github.com/dwyl/html-form-send-email-via-google-script-without-server Which other people end up finding very useful and don't take us that much longer than our original work ...

I would estimate that writing the issues from the REST API to disk would take an extra hour. All we are doing is mirroring the URL path into the FS path and writing a .json file. But the benefit/value this will give to anyone else trying to backup their GitHub is immense!

[des-des]

@nelsonic I am with you. I agree that this would be useful to have.

Once we have build this, we can build this other thing quickly and painlessly!

[des-des]

Here is a good example of using the gh api with elixir: https://github.com/edgurgel/tentacat

[harrygfox]

@iteles

Once that is done and feature requests start pouring in, we'll ask @harrygfox to take a look at the UI/UX and go from there.

glad to be of service! 👍

[des-des]

@iteles @nelsonic. Me and Justen have spent more time talking about this. We have adjusted our time estimates above.

[iteles]

@des-des I've just messaged @Jbarget about this too 👍

[ghost]