search

Expensify / App

Welcome to New Expensify: a complete re-imagination of financial collaboration, centered around chat. Help us build the next generation of Expensify by sharing feedback and contributing to the code.
https://new.expensify.com
MIT License
3.29k stars 2.73k forks source link

[Tracking] ProposalPolice™ 👮🏼 #35108

Open roryabraham opened 7 months ago

roryabraham commented 7 months ago

Slack context: https://expensify.slack.com/archives/C01GTK53T8Q/p1706120841703079

Proposal

develop a GitHub 🤖 using GPT-4 capabilities to standardize and monitor proposals.

Strategy

The purpose of the Expensify Open-Source Community is to provide a global community of all the best React Native developers in the world streamlined access to contribute to the Expensify product. In order to attract and retain talented developers, the Expensify Open-Source Community must be inclusive, fair, and provide equitable opportunity to all community members, including newcomers.

Problem

There is a tendency among the open-source community to perform what I call speedrun proposals – a contributor will post an incomplete or otherwise low-quality proposal very quickly in order to be the first proposal, then subsequently edit it multiple times. Sometimes these edits significantly alter the RCA or solution and/or derive from later proposals without clear acknowledgement. When the C+ or Expensify engineer go to review those proposals, they will tend to choose the first proposal of sufficient quality that solves the problem – and in the case of edited proposals it takes some extra sleuthing to understand the true timeline of proposals. In short, this practice of speedrun proposals inhibits open collaboration amongst contributors and makes it harder for reviewers to fairly evaluate proposals against each other.

Solution

Create a open-source GitHub bot (tentatively named the proposal-police™ 👮🏼🚨) that helps to enforce the documented best practices for proposals, and aids the reviewers in maintaining a clearer timeline of proposals.

More specifically, the proposal-police™ would leverage GPT-4 to:

In conclusion, the proposal-police™ will help promote higher quality proposals and a more informed and trust-based review process :handshake:, improving baseline fairness and allowing community members and Expensify employees to better focus on getting shit :done:!

Issue OwnerCurrent Issue Owner: @ikevin127
melvin-bot[bot] commented 1 month ago

Reviewing label has been removed, please complete the "BugZero Checklist".

melvin-bot[bot] commented 1 month ago

The solution for this issue has been :rocket: deployed to production :rocket: in version 9.0.14-6 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2024-08-07. :confetti_ball:

For reference, here are some details about the assignees on this issue:

melvin-bot[bot] commented 1 month ago

Reviewing label has been removed, please complete the "BugZero Checklist".

melvin-bot[bot] commented 1 month ago

The solution for this issue has been :rocket: deployed to production :rocket: in version 9.0.16-8 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2024-08-13. :confetti_ball:

For reference, here are some details about the assignees on this issue:

melvin-bot[bot] commented 4 weeks ago

The solution for this issue has been :rocket: deployed to production :rocket: in version 9.0.17-2 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2024-08-14. :confetti_ball:

For reference, here are some details about the assignees on this issue:

melvin-bot[bot] commented 3 weeks ago

Payment Summary

[Upwork Job]()

BugZero Checklist (@mallenexpensify)

roryabraham commented 1 week ago

I did a lot of reviews for this one so I'm co-assigning myself

mallenexpensify commented 2 days ago

@ikevin127 , what's status here? Anything else we're waiting on?

ikevin127 commented 2 days ago

@mallenexpensify Hey, as mentioned in this expensify comment:

🟢 Overall the bot's activity looks really good, it's doing its job as instructed and polices proposals for new comments (checking for proposal template match) and for significantly edited proposals.

The latest discussion regarding additional work / changes in code on the proposal police is this slack comment ragarding some issues with the date on a small percentage of proposals that were had significant changes in edits and the date outputted by the proposal police bot, which I plan to work on whenever I get up to date with current PRs / issues.

^ This is not a disruptive issue in any way as, in case the wrong (same 2023 date) is posted by the bot when the proposal is edited significantly, the C+ can click on the comment's edit history and see the edit before the bot's edit and get the correct date - which I'm sure most C+ already do because of the UTC format which we chose for the bot output date.

Other than that one date-related issue which happens every now an then which I plan to address in the near future, I think we're good here for moving forward with the final checklist of requirements before we can proceed to payment.

mallenexpensify commented 1 day ago

Thanks @ikevin127

the final checklist of requirements before we can proceed to payment.

What are the requirements we need to ✅ ? For payment, I think we discussed finishing the project first, then discussing payment. If so, what do you propose for the amount? (let's only focus on this core project and not any future work for deciding the payment amount).

ikevin127 commented 1 day ago

@mallenexpensify By requirements I was referring to things like waiting a bit after proposal police is live to give it some time and show success before discussing payment - I think that happened and we can move forward here.

♻️ Summary

⏳ Timeline of events

🟢 Present day: Overall the bot's activity good, it's doing its job as instructed and polices proposals for new comments (checking for proposal template match) and for significantly edited proposals.

🤖 Some recent examples of Proposal Police in action

CASE 1 - Proposal template match check

CASE 2 - Proposal edited w/ significant changes

💰 Payment amount

Given the amount of work involved to bring this from a 📓 P/S state to 🔴 live and considering the somewhat standard amount paid to C+ for completing projects, I think $2000 would be a fair amount as payment for this project.