[HOLD for payment 2023-08-24] Show chats in the LHN that have at least 1 ADDComment action, or 1 draft message

[JmillsExpensify]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Go to https://staging.new.expensify.com/
2. Login with any account
3. Search for a user that you don't have any messages with, open the chat but don't send any message

Expected Result:

Chat should not show in the LHN as there is not at least 1 ADDComment action, or 1 draft message.

Actual Result:

Empty chat shows in the LHN and remains in the LHN after simply navigating to it.

Workaround:

None

Platforms:

Which of our officially supported platforms is this issue occurring on?

• [x] Android / native
• [x] Android / Chrome
• [x] iOS / native
• [x] iOS / Safari
• [x] MacOS / Chrome / Safari
• [x] MacOS / Desktop

Version Number: 1.2.58-3 Reproducible in staging?: Y Reproducible in production?: Y If this was caught during regression testing, add the test name, ID and link from TestRail: Email or phone of affected tester (no customers): Logs: https://stackoverflow.com/c/expensify/questions/4856 Notes/Photos/Videos: Any additional supporting documentation Expensify/Expensify Issue URL: Issue reported by: Slack conversation:

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01fbd874b8451c4608
• Upwork Job ID: 1618016837618089984
2023-01-24
• Automatic offers:
• | | 0

[MelvinBot]

📣 @bernhardoj You have been assigned to this job by @PauloGasparSv! Please apply to this job in Upwork and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻 Keep in mind: Code of Conduct | Contributing 📖

[PauloGasparSv]

Agree with @thesahindia and their reason, assigned @bernhardoj here

[bernhardoj]

Thanks! I will open the PR in a few hours.

[bernhardoj]

PR is ready for review.

[MelvinBot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[MelvinBot]

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.2.72-1 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

• https://github.com/Expensify/App/pull/15129

If no regressions arise, payment will be issued on 2023-02-23. :confetti_ball:

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• [ ] External issue reporter
• [ ] Contributor that fixed the issue
• [ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[MelvinBot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [ ] [@thesahindia / @PauloGasparSv] The PR that introduced the bug has been identified. Link to the PR:
• [ ] [@thesahindia / @PauloGasparSv] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [ ] [@thesahindia / @PauloGasparSv] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [ ] [@bernhardoj] Propose regression test steps to ensure the same bug will not reach production again.
• [ ] [@JmillsExpensify] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[JmillsExpensify]

Woo! That was fast. PR qualifies for the 50% bonus, so I believe this is where that leaves us:

• Issue reporter: Expensify $0
• Contributor: @bernhardoj $1500 (incl. 50% bonus)
• Contributor+: @thesahindia $1500 (incl. 50% bonus).

Sound right? The upwork job is here: https://www.upwork.com/jobs/~01fbd874b8451c4608. Please apply and we can finalize the final payment when the regression period ends. Thanks all!

[bernhardoj]

Applied.

Here is the Regression Test Steps:

1. Start a new chat with a user that you don't have any messages with
2. Switch to another chat
3. Verify the empty new chat disappear from LHN

Do we agree 👍 or 👎

[PauloGasparSv]

Still need to investigate but maybe the changes from here created https://github.com/Expensify/App/issues/15251

I did test that issue's scenario (sending a comment in the Chat to see if it would stick in the LNH), and it did work so I'm confused. I still didn't replicate it locally and a comment there suggests the problem is in the backend, planning on investigating it on Monday.

[JmillsExpensify]

@bernhardoj I think that addition looks good. I agree with you that everything else should already be covered for the other cases, including an existing chat and a draft message.

[roryabraham]

I just created https://github.com/Expensify/App/pull/15334 to revert the solution for this issue, because:

• it relied on sequence numbers which have been completely deprecated
• it caused a regression

I think this is more fair than proceeding with https://github.com/Expensify/App/issues/15251, because we should not pay a separate bounty to resolve the regression. Instead we should revert the PR that introduced the regression and try again with a proper fix.

[roryabraham]

There is a front-end piece that @bernhardoj can still handle to earn the bounty here, but there's also some back-end stuff we'll need to do. I will start working on writing up a plan of action.

[JmillsExpensify]

Sounds like a plan Rory. I'll update the issue to clarity that we're HOLDing on a regression fix.

[roryabraham]

Okay, it's basically a mini design doc, but here's my proposed plan

---

Proposal

1. Start maintaining an accurate reportActionCount field
2. Don't show DM's with only 1 reportAction (i.e: the CREATED action) in options list, unless it is the currently viewed report

Please re-state the problem that we are trying to solve in this issue.

When you create a new DM but don't send any messages, it should not appear in the LHN for other users. If you don't create a draft message, then navigate away from the chat, it should disappear from the LHN for you.

What is the root cause of that problem?

(after https://github.com/Expensify/App/pull/15334 is merged) There is no special logic in ReportUtils.shouldReportBeInOptionsList to hide DMs with only a created action from the LHN – the default is to show the report.

What changes do you think we should make in order to solve the problem?

Add a reportActionCount field to reports in Onyx

To make this change, we need to examine all NewDot flows that add or remove reportActions from a report. This includes:

• Creating a new report (adds a CREATED action)
• Sending a message or attachment
• Sending an IOU
• Deleting a reportAction

Propagating reportActionCount correctly in the API

First, we need to make sure that when we initially fetch reports, those reports contain their canonical reportActionCount from the back-end. Fortunately, the reportActionCount is already returned from Auth::GetChats, Auth::GetReportSummaryList, Auth::CreateChatReports, etc..., so all we have to do for this is to update ReportUtils::structureReportForOnyx to include reportActionCount, which, as @PauloGasparSv pointed out above, is already used to generate the maxSequenceNumber field.

Next, we'll need to update ReportUtils::buildOnyxDataForNewAction here so that we include reportActionCount as well as the other fields.

Lastly, we'll update ReportAPI::editComment, so that, if we're deleting a comment we always update the report in Onyx to decrement the reportActionCount, regardless for whether it's the newest visible reportAction on the report, or if we shouldKeyReportActionsByID.

Optimistically updating reports with the correct reportActionCount

Creating a new report

There are a few locations where new reports are created in the NewDot front-end, and we can find them by searching for usages of ReportUtils.buildOptimisticChatReport:

• In src/libs/actions/Report.navigateToAndOpenReport
• In src/libs/actions/Report.addPolicyReport
• In ReportUtils.buildOptimisticWorkspaceChats
• In IOU.requestMoney
• In IOU.createSplitsAndOnyxData
• In IOU.getSendMoneyParams

In each of the above-listed locations, there is some code to add an optimistic CREATED action, along with potentially more actions such as an IOU action. We will want to trace each of these flows and ensure that we set the correct optimistic reportActionCount according to the number of optimistically-generated reportActions before writing the report to Onyx.

• The optimistic reportActionCount can safely be assumed to be 1 in each of these places:
  • Here in openReport
  • here in addPolicyReport
  • For each optimistic report in buildOptimisticWorkspaceChats
• In IOU.requestMoney and IOU.getSendMoneyParams, the optimistic reportActionCount should be incremented by 1 if we are not creating a new report, but should be 2 if we are creating a new report
• IOU.createSplitsAndOnyxData should use the same defaults as IOU.requestMoney, except it will vary for the group DM and for every 1:1 DM associated with the bill split.

Sending a message / attachment

This happens in Report.addActions, where we'll always be able to assume that the report has an existing reportActionCount field. We will increment that field by 1 if there is just text or just an attachment, but 2 if there is both text and an attachment.

Sending an IOU

This is covered by the same flows listed above under Creating a new report. If creating a new report, the correct value for reportActionCount on the new report is 2. If adding an IOU action to an existing report, the reportActionCount should be incremented by 1.

Deleting a reportAction

This flow lives in src/libs/actions/Report.deleteReportComment. The report is optimistically updated here, so we need to update that piece of code to also optimistically decrement the reportActionCount on the report by 1.

Don't show DM's with only 1 reportAction in options list (unless it is the currently viewed report)

To do this, we will draw inspiration from https://github.com/Expensify/App/pull/15129:

1. Add a function called isDirectMessage to src/libs/ReportUtils that returns true if the chatType rNVP on the report is empty, and false if it is not.
2. Update src/libs/ReportUtils.shouldReportBeInOptionList to return false if the report is a direct message and the reportActionCount is 1.

Proposed rollout plan

1. Make the PHP changes to add the reportActionCount field into reports in Onyx. This can probably be 1 PR, but could also be split up. It should be covered by unit tests.
2. [HOLD 1] Optimistically add reportActionCount in openReport
3. [HOLD 1] Optimistically add reportActionCount in addPolicyReport
4. [HOLD 1] Optimsitically add reportActionCount in buildOptimisticWorkspaceChats
5. [HOLD 2] Optimistically add reportActionCount in addActions
6. [HOLD 5] Optimistically add reportActionCount in deleteReportComment
7. [HOLD 6] Optimistically add reportActionCount in requestMoney
8. [HOLD 7] Optimistically add reportActionCount in getSendMoneyParams
9. [HOLD 7] Optimistically add reportActionCount in createSplitsAndOnyxData
10. [HOLD 1-9] Don't show DM's with only 1 reportAction in options list (unless it is the currently viewed report)

---

👏🏼 So given that this is a broad issue with a large scope, I suggest we create a GitHub project or tracking issue to track these changes, with a separate issue for each item in the rollout plan above. This particular issue will basically be the issue for item 10 in the proposed rollout plan above.

Does this sound like a good plan @PauloGasparSv @JmillsExpensify @chiragsalian?

[bernhardoj]

Sorry for the regression guys.

I like the plan above, I can handle the frontend part. I would also like to add that because we will optimistically increment the reportActionCount, we should decrement it back if it fails.

For example, when we send a single message, we will increment it by 1, but when it fails for some reason, we decrement it by 1.

[MelvinBot]

📣 @bernhardoj! 📣

Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork. Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.

Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[bernhardoj]

Contributor details Your Expensify account email: bernhard.josephus@gmail.com Upwork Profile Link: https://www.upwork.com/freelancers/~013e648527fa0266fd

[MelvinBot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[chiragsalian]

I like the proposed plan rory 👍

My one concern is the optimistic plan. In the rollout plan I'm wondering if before step 2 we should create a centralized method like getReportActionCount that connects to the report and returns the reportActionCount. This way all the other places can use this method and then increment or decrement the number as needed.

Let's remember for every optimistic action. We should also add failure data to revert the value as necessary.

I'm not sure i follow why some of the flows are on HOLD for steps other than 1.

And i agree we should make another tracking issue.

[JmillsExpensify]

All good @bernhardoj! Thanks @roryabraham for the investigation and getting this mini-plan typed up.

[JmillsExpensify]

Ya'll want me to take on creating the tracking issue and related issues?

[PauloGasparSv]

The proposal is amazing, thanks so much for the help @roryabraham!

Ya'll want me to take on creating the tracking issue and related issues?

@JmillsExpensify that would be great! Would you mind creating those?

[roryabraham]

I would also like to add that because we will optimistically increment the reportActionCount, we should decrement it back if it fails.

Correct @bernhardoj! When dealing with API.write commands, we have optimisticData, successData and failureData. What you've described is a good use-case for failureData 👍🏼

[JmillsExpensify]

Nice, I'll do that now ya'll.

[JmillsExpensify]

Alright, the tracking issue and related implementation issues are created here: https://github.com/Expensify/App/issues/15352.

[JmillsExpensify]

A couple of questions follow:

• There's a lot of internal implementation, so who's going to take these on? Someone in this issue, or should we raise it in #engineering-chat in Slack? I went ahead and assigned myself for now.
• Shouldn't we post on this more broadly in open-source somewhere?

Oh and finally, perhaps we'll put this issue on hold now for the tracking issue?

Once we align on the above I can clean up some of the labels and what not on the implementation issues. Thanks!

[chiragsalian]

Someone in this issue, or should we raise it in #engineering-chat in Slack?

Anyone can. if we go with the internal route i prefer if we post on slack so that it's spread out faster.

Shouldn't we post on this more broadly in open-source somewhere?

I think all the App issues posted above can be handled externally. We just have to share steps including failure steps with our expected outcome. Maybe @roryabraham can confirm since I'm unsure why he suggested all internal earlier.

Oh and finally, perhaps we'll put this issue on hold now for the tracking issue?

I'd say we can just close this since the last issue mentioned here should tackle this issue. If it hasn't then we reopened this issue later.

Also @roryabraham, did you have a chance to check out my comment asking for

1. Clarification of the HOLD status for the ones not on HOLD for 1. I don't see why they are on HOLD since each could be worked on in parallel as there shouldn't be any impact on the UI except for the last issue. For example, for deleteReportComment we could add to the steps to fake insert reportActionCount 10 and then the expected optimistic data should decrease the number and its failure data should increase the same number. It doesn't feel like the HOLD is necessary except for item 10 since that is user-facing.
2. If we need a step before rollout plan item 2 to create a centralized method so that every step after can make use of it?

[JmillsExpensify]

I'd say we can just close this since the last issue mentioned https://github.com/Expensify/App/issues/15352 should tackle this issue. If it hasn't then we reopened this issue later.

That's cool, it's just that we also need to pay out on this issue, and it's tied to a regression at the moment.

[PauloGasparSv]

Assigning the "Add a reportActionCount field to reports in Onyx" internal issue https://github.com/Expensify/App/issues/15353 to me here if that's ok!

I really like @chiragsalian suggestion to create a centralized function to fetch the report action count (like getReportActionCount) and I think we should add an issue for that.

[PauloGasparSv]

cc @roryabraham

Lastly, we'll update ReportAPI::editComment, so that, if we're deleting a comment we always update the report in Onyx to decrement the reportActionCount, regardless for whether it's the newest visible reportAction on the report, or if we shouldKeyReportActionsByID.

Hey, while testing https://github.com/Expensify/Web-Expensify/pull/36578 I realized that deleting a reportAction doesn't decrease reportActionCount in GetChats.cpp.

IMO that's ok but I want to know your opinions. If that's the case then I don't think we need to update the logic for deleting comments anymore since that counter doesn't decrease right?

[JmillsExpensify]

Still held for the linked tracking issue. Adding weekly back.

[PauloGasparSv]

No updates or blockers here, P.R. still in progress! Had to shift focus to other issues but now I'm coming back to this.

[PauloGasparSv]

Still no updates here.

[alex-mechler]

https://github.com/Expensify/App/issues/15952 is a dupe of this

[PauloGasparSv]

This time I have an update :D

I've just finished implementing the Web-E and Auth P.R.'s for adding reportActionCount in those layers. I'll finish fixing and adding tests then they'll be ready for reviews. I believe this will unblock all the other issues we have in the main tracking one.

---

Changing this to monthly or else this will keep getting overdue!

[marcaaron]

Just out of curiosity, what do we still use reportActionCount for? Since we don't have sequenceNumber is there any value in it?

[marcaaron]

I thought we mainly just used it to calculate the unreads, but now that is something more like "last read compared to most recent created".

[trjExpensify]

@JmillsExpensify can we update the title to point to the issue it's held on for clarity? Pretty sure it's this one: https://github.com/Expensify/App/issues/15352.

[PauloGasparSv]

@JmillsExpensify can we update the title to point to the issue it's held on for clarity? Pretty sure it's this one: https://github.com/Expensify/App/issues/15352.

I think this is the revert P.R. https://github.com/Expensify/App/pull/15334 and this was the regression https://github.com/Expensify/App/issues/15251

I don't think this is on HOLD anymore.

Following this thread and this comment I'm trying to find time this week to test the feature using the lastMessageHTML field instead of the reportActionCount.

[JmillsExpensify]

Agreed, I don't think this is one hold. This got wrapped up into the larger reportActionCount project.

[PauloGasparSv]

I'll have time to implement this between this and next week so I'll have an update here soon.

[michaelhaxhiu]

Since this is no longer on Hold, I'm going to flip it back to Daily per the standardized process (I'm just combing through for the weekly resync).

[PauloGasparSv]

Sure thing @michaelhaxhiu! No updates since this and this is still on my todo list.

[JmillsExpensify]

@roryabraham For confirmation, where did we land with implementing reportActionCount. I kind of remember a conversation where it might be paused?

[melvin-bot[bot]]

@JmillsExpensify, @PauloGasparSv, @bernhardoj, @thesahindia Whoops! This issue is 2 days overdue. Let's get this updated quick!

[PauloGasparSv]

No updates here because I couldn't focus on this issue last week as I planned. Now planning to work on this before EOW.

[PauloGasparSv]

No updates but we discussed this issue today here.

[PauloGasparSv]

No updates

[PauloGasparSv]

Will start working on this tomorrow!