angelasjin commented 2 years ago

Following some conversations with Milana (Documentation team rep), there are a few suggested contributions for tracking:

In the Documentation team Github, track issues and assignments, possibly commenters.
Tracking code examples or user contributed notes, both for the submitter and the reviewer

If possible, the team would like to be able to assign badges automatically for these types of contribution (in addition to being able to manually assign them.

iandunn commented 2 years ago

In the Documentation team Github, track issues and assignments, possibly commenters.

Opening/closing issues are already tracked (as of a few months ago), but I'm guessing comments would be easy. cc @dd32

Tracking code examples or user contributed notes, both for the submitter and the reviewer

👍🏻

iandunn commented 2 years ago

For GitHub activity to show up, though, folks need to connect their w.org accounts to their GitHub account.

Not everyone knows that, so maybe it's something we should make more visible somehow.

courtneyr-dev commented 2 years ago

Make visible

Lesson Plan about creating a .org account. We could elevate this to ALL teams for contributor days.

Would you like https://learn.wordpress.org/workshop/set-up-a-wordpress-org-account/ to cover it as well, or perhaps we can make a separate lesson plan and workshop about each?

We're working on courses around getting started as a contributor, and this would be super beginner-oriented. Folks should know some of this before even leaving issues in the forums.

dd32 commented 2 years ago

When it comes to GitHub activity, I'd prefer to apply recorded events equally to all GitHub repo's, rather than specific rules for specific repo's / teams.

In the Documentation team Github, track issues and assignments, possibly commenters.

Opening/closing issues are already tracked (as of a few months ago), but I'm guessing comments would be easy.

Tracking Assignment of issues shouldn't be an issue, I'm hesitant to track Comments though due to the sheer number of comments made.

Perhaps this is one of the locations where comments tracking would be worthwhile though?

iandunn commented 2 years ago

I'm hesitant to track Comments though due to the sheer number.

That's also going to be a problem when we start tracking translations. One idea for that is to add something like Wrote 15 comments in 3 GitHub repositories this week, rather than 15 individual activity items. GitHub might provide a permalink we could include.

Or maybe we add individual items in the database, but then on the front end we display them all with something like <details>, for a better UX:

<details>
    <summary>Wrote 15 comments in 3 GitHub repositories this week</summary>

    <ul>
        <li>Wrote a comment on issue #371 in the WordPress/wporg-news-2021 repository:
            lorum ipsum blah blah blah
        </li>

        <li>Wrote a comment on issue #5432 in the WordPress/gutenberg repository:
            lorum ipsum blah blah blah
        </li>

        <li>Wrote a comment on issue #2353 in the WordPress/Documentation-Issue-Tracker repository:
            lorum ipsum blah blah blah
        </li>
    </ul>
</details>

We add an individual item for forum replies and p2 comments, though, which seems analogous. Are GH/Trac comments significantly more frequent?

iandunn commented 2 years ago

Would you like https://learn.wordpress.org/workshop/set-up-a-wordpress-org-account/ to cover it as well, or perhaps we can make a separate lesson plan and workshop about each?

I don't have a strong opinion about how, but I think mentioning it is a great idea 👍🏻

zzap commented 2 years ago

I'm hesitant to track Comments though due to the sheer number of comments made.

Fighting with all the notifications every day, I 100% agree with this. On the other hand, Documentation Issues Tracker repo has little to no other activity. We use that repo only for tracking issues thus comments being almost the only contribution.

We also saw a lot of contributions hunting for badges in past and that's the reason why I'm not for automating tracking contributions just by posting a comment. Some kind of manual action here would be appreciated even though nothing as a possible solution comes to mind.

In a perfect world we would have some kind of Team dashboard at wp.org, where we could see all tracked contributions from all places and a button "Confirm" next to each :heart_eyes:

dd32 commented 2 years ago

Or maybe we add individual items in the database, but then on the front end we display them all with something like <details>, for a better UX:

That seems like a reasonable idea to me, it'll require some pretty severe changes to the Profiles querying method though.

We add an individual item for forum replies and p2 comments, though, which seems analogous. Are GH/Trac comments significantly more frequent?

I don't get the feeling that P2s are all that active on WordPress.org unfortunately, make.wordpress.org/core would probably be the most active one I would've assume, but there's only been 127 comments by 89 people since 2021-04-01 (26 days), 3-5 entries per day in that period.

In the same time, we've had ~2.5k GitHub activity entries, and that's just from Issues & PRs open/closing.

iandunn commented 2 years ago

it'll require some pretty severe changes to the Profiles querying method though.

I was thinking something like:

Start inserting comments into wporg_github_activity
Add an index on the category column
Add a WHERE category <> 'comment' to the query in WPORG_Profiles::get_code()
Add a cron job that runs once a week and:
- Selects all comment items in the past week
- Loops through the query results and re-indexes them by user (i.e., $comments[ $user ][] = $comment;
- Loops through the indexed array and adds a single activity entry with the <details> markup for each user
Modify WPORG_Profiles::get_activities() to not truncate content when it contains <details>

Do you think it'd be more complicated than that?

A similar approach could be used for something like Wrote 15 comments in 3 GitHub repositories this week, if we don't want the details.

dd32 commented 2 years ago

Do you think it'd be more complicated than that?

Well, I was thinking that you'd want the activity items to show instantly, not be delayed by a week.. I didn't really think of adding it to a cron instead. I still don't think I like the cron approach.

My complicated thing was that I figured we'd just be querying in realtime, not cron'ing it, and in that case the 50 limit and max 10 items logic in the activity stream would result in yesterdays items falling off the screen very quickly, even though it's including last weeks activity for other items.

iandunn commented 2 years ago

I was thinking that you'd want the activity items to show instantly, not be delayed by a week.

Fair point. We could create a rolling digest, instead of a static one.

User foo hasn't posted any comments this week, and doesn't have any comment activity for this week on their profile
They post a comment
A cron is running every 15 minutes, and it sees the new comment. Since there aren't any others this week, it creates an activity entry in the BuddyPress database.
Later they post a 2nd comment
The next time the cron runs, it updates the existing activity entry, which now says they've posted 2 comments this week

StevenDufresne commented 2 years ago

In the Documentation team Github, track issues and assignments, possibly commenters.

We are currently not tracking assignments, though as @dd32 has mentioned it's easy to turn on. But, if we do track comments, and it appears as we will.... Are assignments still useful? You can be assigned by someone else but never end up doing anything.

StevenDufresne commented 2 years ago

Summarizing where I think this ticket is:

Tracking code examples or user-contributed notes, both for the submitter and the reviewer

Will do, not implement

Do we need assignments?

Needs Decision

Should we add comments?

Needs Decision

Yes
- How do we reduce noise?
- Cron Job
- How do we limit badge hunting?

I tend to agree that GitHub activity should be shared across all teams/repos and should it not work for some teams, we could exclude them. If that is a fair assumption, could we move the comments discussion & assignments to separate ( or the same ) tickets? That would leave us with a clearer scope on this ticket and we could action the first item which is documentation team specific.

iandunn commented 2 years ago

if we do track comments .... Are assignments still useful?

I would guess that they're not as important, since the comments are the actual contribution. Or are assignments itself a significant contribution, since it represents managing & delegating workflow on the Docs team; @zzap ?

How do we reduce noise?

197 will solve that

How do we limit badge hunting

The scope of this milestone focuses on activity rather than badges, so I think we can leave that discussion to another day

move the comments discussion & assignments to separate ( or the same ) tickets?

Either that, or add checklists to this ticket. I don't have an opinion as long as it's obvious what's been done and what hasn't

StevenDufresne commented 2 years ago

Tracking code examples or user-contributed notes, both for the submitter and the reviewer Partially implement in https://meta.trac.wordpress.org/changeset/11934.

This will turn on comments for developer.wordpress.org. However, there are no code paths that credit the reviewer at the moment. Do we want to turn that on for all reviewers? It seems like that type of action should be credited consistently.

iandunn commented 2 years ago

Tracking reviews seems like a good idea. It might be something we want to use digests for (#197). The current code doesn't handle bulk actions yet (like approving 20 translations, or 20 devhub comments), though. I'm working on that for #196. It may be good to wait and see how that turns out.

iandunn commented 2 years ago

The bulk process is working now:

Notifier: https://github.com/WordPress/wordpress.org/commit/fbade8715ec39999628d5330481f6ef41f35ef25 Handler: https://github.com/WordPress/wordpress.org/commit/564088b2a40f75584a8c80a02cdda3959c37e3f3

zzap commented 2 years ago

if we do track comments .... Are assignments still useful?

I would guess that they're not as important, since the comments are the actual contribution. Or are assignments itself a significant contribution, since it represents managing & delegating workflow on the Docs team; @zzap ?

I'm struggling with that one to be honest. Assignments are significant but sometimes we want people who comment to get attribution as well. But not for all comments, especially because that's an easy way for badge hunting.

If it's possible to track it the same way Trac does - certain people comment props to... and that gets tracked, that would be ideal. If not, I guess we'll have to do it manually.

WordPress / five-for-the-future

Tracking Contributions: Make Documentation team #178

Tracking code examples or user-contributed notes, both for the submitter and the reviewer

Do we need assignments?

Should we add comments?

197 will solve that