search

Expensify / App

Welcome to New Expensify: a complete re-imagination of financial collaboration, centered around chat. Help us build the next generation of Expensify by sharing feedback and contributing to the code.
https://new.expensify.com
MIT License
3.53k stars 2.88k forks source link

[HOLD for payment 2024-11-07] QBO - Syncing is stuck at "Waiting for imported data to load" #49371

Open IuliiaHerets opened 1 month ago

IuliiaHerets commented 1 month ago

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

Version Number: 9.0.37-0 Reproducible in staging?: Y Reproducible in production?: N Email or phone of affected tester (no customers): applausetester+kh010901@applause.expensifail.com Issue reported by: Applause Internal Team

Action Performed:

  1. Go to staging.new.expensify.com
  2. Go to workspace settings > Accounting.
  3. Connect to QBO.
  4. Enter credential and log in.

Expected Result:

Syncing will be completed without issue.

Actual Result:

Syncing is stuck at "Waiting for imported data to load". It disappears after reopening Accounting tab.

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

Screenshots/Videos

https://github.com/user-attachments/assets/64c85a7e-af00-4adb-b683-2f164c978928

View all open jobs on GitHub

Issue OwnerCurrent Issue Owner: @slafortune
melvin-bot[bot] commented 1 month ago

Triggered auto assignment to @slafortune (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

IuliiaHerets commented 1 month ago

We think that this bug might be related to #wave-collect - Release 1

etCoderDysto commented 1 month ago

Is there a credential for contributor to connect to QBO?

melvin-bot[bot] commented 1 month ago

Triggered auto assignment to @madmax330 (DeployBlockerCash), see https://stackoverflowteams.com/c/expensify/questions/9980/ for more details.

github-actions[bot] commented 1 month ago

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open `StagingDeployCash` deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

  1. Identify the pull request that introduced this issue and revert it.
  2. Find someone who can quickly fix the issue.
  3. Fix the issue yourself.
IuliiaHerets commented 1 month ago

Forget to add a DeployBlockerCash label

izarutskaya commented 1 month ago

This issue reproducible for Xero connection also. Let me know if we need to create a new GH ticket. Thank you

https://github.com/user-attachments/assets/8bc748ce-8f98-4622-83fd-456cc5054e84

slafortune commented 1 month ago

I was able to successfully connect to SuccessTeam1 - In the video I don't see where you entered the credentials, just selected a company file -

https://github.com/user-attachments/assets/66c9bbf4-3370-461f-babb-0d7815c5c44a

grgia commented 1 month ago

I got stuck on waiting for data to load but it loaded on refresh

image
grgia commented 1 month ago

I think we can at the very least demote this for now.

arosiclair commented 1 month ago

I only see these logs for applausetester+kh010901@applause.expensifail.com. The auto sync failed due to 

QBO auto sync error ~~ msg: 'Refresh token is no longer valid, need to renew credentials.' code: '407' class: 'OAuthAuthenticationException'

which doesn't really match the bug report (there was no error it just appeared stuck). That was also from yesterday not today. So it looks like the email in the report is not accurate.

Syncing is stuck at "Waiting for imported data to load". It disappears after reopening Accounting tab.

This sounds like some sync progress updates were not received. IIRC those don't use reliable updates so that may fix that issue. cc @aldo-expensify

aldo-expensify commented 1 month ago

This sounds like some sync progress updates were not received. IIRC those don't use reliable updates so that may fix that issue. cc @aldo-expensify

Seems like the jobDone update is the only progress update sent through reliable updates: https://github.com/Expensify/Integration-Server/blob/f0fde768b3ea60545eac564ba367bf09cff67f40/src/expensify/progress/PushOnyxUpdateProgressManager.java#L51-L53

madmax330 commented 1 month ago

@aldo-expensify @arosiclair can one of you pick this up since you seem to know what's going on?

arosiclair commented 1 month ago

@aldo-expensify I think you're probably best to take it on?

aldo-expensify commented 1 month ago

I haven't had time yet to investigate, I'll try to get to it by the end of the week

melvin-bot[bot] commented 1 month ago

@slafortune @aldo-expensify this issue was created 2 weeks ago. Are we close to a solution? Let's make sure we're treating this as a top priority. Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!

aldo-expensify commented 1 month ago

From the reports, it looks like a case where we miss onyx updates. I have tried a few times to reproduce and I haven't been able:

https://github.com/user-attachments/assets/c0c62e7b-7fab-4744-bf27-95afa5406d8c

Besides what you see in the video, I also tested:

MelvinBot commented 1 month ago

This has been labelled "Needs Reproduction". Follow the steps here: https://stackoverflowteams.com/c/expensify/questions/16989

aldo-expensify commented 1 month ago

Oh, trying again with closing the tab while the sync is happening left it in a permanent loading state

image

The connections data in the policy is missing:

image

The policySyncProgress_ onyx data is showing that the job finished:

image

so the problem seems to be that somehow we ended up missing the policy.connections data.

I then went offline and back online and it still shows as loading, but now the connections data is there:

image

Even if now the policy.connections data is there, the spinner is still stuck there... this last thing sounds like the component did not rerender when the onyx data changed.

image
aldo-expensify commented 3 weeks ago

I'm not sure this is necessarily internal, from what I found here, it looks like we may be not re-rendering correctly when onyx data changes.

aldo-expensify commented 3 weeks ago

I fixed my dev env so I have integrations and pushed events working again. I'll try to reproduce again and see if I can fix something.

trjExpensify commented 3 weeks ago

I'm not sure this is necessarily internal,

Cool, if we firm up on that, I'm sure someone from the C+ pool would be willing to do the work if they can.

aldo-expensify commented 3 weeks ago

Two things I see for now:

aldo-expensify commented 3 weeks ago

Regarding the jobDone being a reliable update, we had a bug here, fixing here: https://github.com/Expensify/Auth/pull/12801

aldo-expensify commented 3 weeks ago

Easy reproduction steps:

  1. Connect QBO to a workspace
  2. Close the tab before the connection finishes syncing
  3. Wait for a minute or more until the connecting syncing has finished
  4. Reopen the tab
  5. You will be left with the spinner showing like it is syncing even if it finished

Why is this happening? because of point 2 here: https://github.com/Expensify/App/issues/49371#issuecomment-2418305801

The PR https://github.com/Expensify/Auth/pull/12801 was deployed and it was a necessary fix, but we still need to do something about commands like OpenInitialSettingsPage not loading missed onyx updates. I'll start a slack convo in the engineering chat after lunch to get some consensus on what to do for this case.

Asked here: https://expensify.slack.com/archives/C03TQ48KC/p1729204325866909

aldo-expensify commented 3 weeks ago

Seems like the expectation is that ReconnectApp should have been called after reopening the tab, so I'm going to investigate why it is not happening.

aldo-expensify commented 3 weeks ago

PR up: https://github.com/Expensify/App/pull/51110 for fixing the missing calls to ReconnectApp.

@abdulrahuman5196 got assigned as reviewer (C+)

slafortune commented 1 week ago

Just deployed to staging

melvin-bot[bot] commented 1 week ago

Reviewing label has been removed, please complete the "BugZero Checklist".

melvin-bot[bot] commented 1 week ago

The solution for this issue has been :rocket: deployed to production :rocket: in version 9.0.55-10 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2024-11-07. :confetti_ball:

For reference, here are some details about the assignees on this issue:

melvin-bot[bot] commented 1 week ago

@abdulrahuman5196 @slafortune The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the BugZero Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]

slafortune commented 1 week ago

BugZero Checklist:

Bug classification Source of bug: - [ ] 1a. Result of the original design (eg. a case wasn't considered) - [ ] 1b. Mistake during implementation - [ ] 1c. Backend bug - [ ] 1z. Other: Where bug was reported: - [ ] 2a. Reported on production - [ ] 2b. Reported on staging (deploy blocker) - [ ] 2c. Reported on a PR - [ ] 2z. Other: Who reported the bug: - [ ] 3a. Expensify user - [ ] 3b. Expensify employee - [ ] 3c. Contributor - [ ] 3d. QA - [ ] 3z. Other:
Regression Test Proposal Template - [ ] **[BugZero Assignee]** Create a GH issue for creating/updating the regression test once above steps have been agreed upon. Link to issue: ## Regression Test Proposal ### Precondition: - ### Test: 1. Do we agree 👍 or 👎
slafortune commented 2 days ago

@abdulrahuman5196 can you please complete the checklist?

abekkala commented 1 day ago

bump @abdulrahuman5196

The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the BugZero Checklist from here into a new comment on this GH and complete it.