[HOLD for payment 2024-03-07] [$1000] IOU-The receipt didn't upload error message appears when create Scan expense with pdf

[kavimuru]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Version Number: 1.4.24-4 Reproducible in staging?: y Reproducible in production?: y If this was caught during regression testing, add the test name, ID and link from TestRail: Email or phone of affected tester (no customers): Logs: https://stackoverflow.com/c/expensify/questions/4856 Expensify/Expensify Issue URL: Issue reported by: Applause internal team Slack conversation:

Action Performed:

1. Launch app
2. Tap on a Workspace chat
3. Tap plus icon --- request money --- Scan---Choose document
4. Upload a pdf file which has a space in file name (eg: boat one)
5. Tap request
6. Tap on scan expense created to open details page
7. Tap on receipt and note pdf can be viewed
8. Tap back button
9. Enter all the fields
10. Tap "from" user link and navigate to workspace chat

Expected Result:

"The receipt didn't upload.." error message must not be appeared when user create Scan expense with PDF file.

Actual Result:

"The receipt didn't upload.." error message appears when create Scan expense with PDF file.

Workaround:

unknonw

Platforms:

Which of our officially supported platforms is this issue occurring on?

• [x] Android: Native
• [ ] Android: mWeb Chrome
• [ ] iOS: Native
• [ ] iOS: mWeb Safari
• [ ] MacOS: Chrome / Safari
• [ ] MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

https://github.com/Expensify/App/assets/43996225/8304394e-3c8c-4926-8009-f89c365518d8

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~017a49b283451a8f08
• Upwork Job ID: 1745750282523729920
• Last Price Increase: 2024-02-05
• Automatic offers:
• situchan | Contributor | 28105886

Issue Owner

Current Issue Owner: @laurenreidexpensify

[melvin-bot[bot]]

Triggered auto assignment to @laurenreidexpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot[bot]]

Job added to Upwork: https://www.upwork.com/jobs/~017a49b283451a8f08

[melvin-bot[bot]]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @thesahindia (External)

[ikevin127]

Proposal

Please re-state the problem that we are trying to solve in this issue

"The receipt didn't upload.." error message appears when create Scan expense with PDF file.

What is the root cause of that problem?

Note: this only happens on Android for files which have empty spaces in their names.

In .../AttachmentPicker/index.native.js within the getDataForUpload function in fileResult object we pass uri: fileData.fileCopyUri || fileData.uri and on Android the sources look like this:

• fileData.fileCopyUri file:///data/user/0/com.expensify.chat.dev/cache/e22200b8-5380-47ea-9fea-f57ca9ad5b8d/cb%20copy.pdf
• fileData.uri content://com.android.providers.media.documents/document/document%3A1000000037

currently the first one is used on Android which is a cached reference to the file which while it does work for preview purposes, when the RequestMoney call is made with the cached source the request fails right away because the path doesn't contain the actual file.

RequestMoney - Fail Response

```json { "message": "Network request failed", "request": { "command": "RequestMoney", "data": { "debtorEmail": "kevin.bader96+7@gmail.com", "debtorAccountID": 15939231, "amount": 0, "currency": "RON", "comment": "", "created": "2024-01-12", "merchant": "(none)", "iouReportID": "4956625719563169", "chatReportID": "1962451004289871", "transactionID": "2546113264902528855", "reportActionID": "764800307773977761", "createdChatReportActionID": 0, "createdIOUReportActionID": 0, "reportPreviewReportActionID": "4677077960755453582", "receipt": { "_data": { "blobId": "b82d40bb-43a0-4e8f-a753-b79ea96e429b", "offset": 0, "size": 594360, "type": "", "__collector": {}, "name": "cb_copy.pdf" }, "source": "file:///data/user/0/com.expensify.chat.dev/cache/1a1901f7-e5ec-4ffd-91b2-abbc8dce0f66/cb%20copy.pdf", "uri": "file:///data/user/0/com.expensify.chat.dev/cache/1a1901f7-e5ec-4ffd-91b2-abbc8dce0f66/cb%20copy.pdf", "state": "SCANREADY" }, "receiptState": "SCANREADY", "taxCode": "", "taxAmount": 0, "billable": false, "appversion": "1.4.24-7", "apiRequestType": "write", "pusherSocketID": "640434.130102", "shouldRetry": true, "canCancel": true }, "successData": [{ "onyxMethod": "merge", "key": "report_4956625719563169", "value": { "pendingFields": null, "errorFields": null } }, { "onyxMethod": "merge", "key": "transactions_2546113264902528855", "value": { "pendingAction": null, "pendingFields": null } }, { "onyxMethod": "merge", "key": "reportActions_1962451004289871", "value": { "4677077960755453582": { "pendingAction": null } } }, { "onyxMethod": "merge", "key": "reportActions_4956625719563169", "value": { "764800307773977761": { "pendingAction": null, "errors": null } } }], "failureData": [{ "onyxMethod": "merge", "key": "report_1962451004289871", "value": { "iouReportID": 4956625719563169, "lastReadTime": "2024-01-12 19:11:15.349", "pendingFields": null } }, { "onyxMethod": "merge", "key": "report_4956625719563169", "value": { "pendingFields": null, "errorFields": {} } }, { "onyxMethod": "merge", "key": "transactions_2546113264902528855", "value": { "errors": { "1705087515596000": "iou.error.genericCreateFailureMessage" }, "pendingAction": null, "pendingFields": null } }, { "onyxMethod": "set", "key": "transactionsDraft_1", "value": null }, { "onyxMethod": "merge", "key": "reportActions_1962451004289871", "value": { "4677077960755453582": { "created": "2024-01-12 19:25:15.595", "errors": { "1705087515596000": { "error": "receiptError", "source": "file:///data/user/0/com.expensify.chat.dev/cache/1a1901f7-e5ec-4ffd-91b2-abbc8dce0f66/cb%20copy.pdf", "filename": "cb_copy.pdf" } } } } }, { "onyxMethod": "merge", "key": "reportActions_4956625719563169", "value": { "764800307773977761": { "errors": { "1705087515596000": { "error": "receiptError", "source": "file:///data/user/0/com.expensify.chat.dev/cache/1a1901f7-e5ec-4ffd-91b2-abbc8dce0f66/cb%20copy.pdf", "filename": "cb_copy.pdf" } } } } }, { "onyxMethod": "set", "key": "transactionViolations_2546113264902528855", "value": [] }] } } DEBUG[info][NetworkConnection] recheck NetInfo - "" DEBUG[info][NetworkConnection] NetInfo state change - { "details": { "isConnectionExpensive": false, "txLinkSpeed": 2, "rxLinkSpeed": 2, "linkSpeed": 2, "subnet": "255.255.255.0", "ipAddress": "10.0.2.16", "frequency": 2447, "strength": 99, "bssid": "02:00:00:00:00:00" }, "isConnected": true, "type": "wifi", "isInternetReachable": true, "isWifiEnabled": true } ```

What changes do you think we should make in order to solve the problem?

Using fileData.uri fixes the issue of the RequestMoney call failing because on Android fileData.uri is the real path to the actual file instead of a cached reference path.

Given that the issue is present only on Android, we can either use Platform.select with android changed to fileData.uri only while maintaining fileData.fileCopyUri || fileData.uri for ios where the issue is not present. Or create a custom library @lib where we will have index.ts noop and index.android.ts and index.ios.ts that return the fileResult.uri accordingly.

Videos

Android: Native

https://github.com/Expensify/App/assets/56457735/d7132d24-283c-4d4f-ba87-1fb2da52f1dc

[laurenreidexpensify]

Not overdue, Melvin being a bit harsh with the weekend clock

[thesahindia]

Won't be able to take this. Please reassign.

[situchan]

I am interested in reviewing this

[melvin-bot[bot]]

📣 @situchan 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link Upwork job Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻 Keep in mind: Code of Conduct | Contributing 📖

[laurenreidexpensify]

@situchan ready for review thanks

[situchan]

@ikevin127 thanks for the proposal. Your RCA doesn't explain why this bug happens only on files which have spaces.

[ikevin127]

@situchan The root cause of the issue only happening on Android for files which have spaces in their names is that the fileCopyUri cache path of the file with spaces is url encoded which replaces empty spaces with %20 instead of empty space ` which when passed to theRequestMoney` endpoint, the path is corrupt.

https://github.com/Expensify/App/blob/a8acf44090c9aab4e9af7967a79758472ada8fc3/src/components/AttachmentPicker/index.native.js#L74

Example: if in the function getDataForUpload before the if's we add:

RNFetchBlob.fs.stat(decodeURIComponent(fileData.fileCopyUri)).then((stats) => console.log('STATS', stats)).catch((error) => console.log('ERROR', error));

we can see that we get the STATS console.log('STATS', stats) and not ERROR if we decode the fileData.fileCopyUri compared to running the code block without decoding it which throws error: failed to stat path '/data/user/0/com.expensify.chat.dev/cache/3ab77e4f-9f84-4fa7-8566-8a7a4a2fec45/cb%20copy.pdf' because it does not exist or it is not a folder.

Even though I could confirm this, sending the decoded fileData.fileCopyUri to the RequestMoney request, the request still fails. But using fileData.uri absolute path on Android fixes the issue.

This is all I can say as to why this edge case when using fileCopyUri cached path vs uri absolute path to the actual file. I could not find any explanation of this edge case anywhere online nor could I find any source code which would explain why this is the case.

[melvin-bot[bot]]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[laurenreidexpensify]

Still waiting for proposals

[melvin-bot[bot]]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[laurenreidexpensify]

@situchan could you review the feedback above?

[situchan]

The root cause is still not clear from the last comment above. I was looking for the solution with keeping fileCopyUri || uri as this already works perfect on iOS.

[ikevin127]

Indeed it does work on iOS - meaning this is an Android specific issue for which I wasn't able to find more context to explain the root cause as to why this edge case is happening for files with spaces in their names when using fileCopyUri on Android: Native 🙁

What we can do to solve this issue while maintaining the current iOS specific logic unchanged, as mentioned in the solution:

create a custom library @lib where we will have index.ts noop and index.android.ts and index.ios.ts that return the fileResult.uri accordingly.

meaning for Android we return the uri whereas for iOS we return fileData.fileCopyUri || fileData.uri just like we currently do for both.

If this is enough in order to move forward with fixing the issue, I'm available to raise a PR immediately 🚀

[melvin-bot[bot]]

@laurenreidexpensify, @situchan Whoops! This issue is 2 days overdue. Let's get this updated quick!

[situchan]

still looking for alternative solution

[melvin-bot[bot]]

@laurenreidexpensify @situchan this issue is now 3 weeks old. There is one more week left before this issue breaks WAQ and will need to go internal. What needs to happen to get a PR in review this week? Please create a thread in #expensify-open-source to discuss. Thanks!

[melvin-bot[bot]]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[melvin-bot[bot]]

Upwork job price has been updated to $1000

[laurenreidexpensify]

increased bounty

[rojiphil]

What is the root cause of that problem?

How does scan receipt get uploaded? At the scan step, we use picker to open an attachment. Once the attachment is available, setReceiptAndNavigate is called here for native devices to set the receipt details in the transaction draft and, further, navigate to the confirmation step. Now, on component mount of the confirmation step, we check if the scan receipt is available here. And, if available, we set the receipt File object setReceiptFile which is later used in money request generation here. The API request would add the receipt File object via FormData which will trigger the upload of receipt file

What is going wrong here? Everything goes well for Android native devices until the check for scan receipt availability here in the confirmation step. navigateToStartStepIfScanFileCannotBeRead internally makes use of readFileAsync here to fetch the receipt File object. However, when the file name contains space on android native devices, the File type is empty here which results in failure of receipt upload operation and the resultant error message.

What changes do you think we should make in order to solve the problem?

To fix this, we can set the File type correctly here. Reading the file and determining the correct type works fine even when space is present within file name here. So, we can preserve the identified File type within transaction’s receipt here and use the type in confirmation step like this:

Add receiptType here const receiptType = lodashGet(transaction, 'receipt.type');

Pass receiptType to navigateToStartStepIfScanFileCannotBeRead here IOU.navigateToStartStepIfScanFileCannotBeRead(receiptFilename, receiptPath, onSuccess, requestType, iouType, transactionID, reportID, receiptType);

Pass receiptType to readFileAsync here FileUtils.readFileAsync(receiptPath, receiptFilename, onSuccess, onFailure, receiptType);

Finally, set the File type if blob.type is not valid whereas we could still retrieve the blob itself here const file = new File([blob], cleanFileName(fileName), {type: blob.type ? blob.type : fileType});

Additionally, we also need to pass receiptType to navigateToStartStepIfScanFileCannotBeRead in IOURequestStepParticipants as we do for IOURequestStepConfirmation to support the global FAB flow here

What alternative solutions did you explore? (Optional)

[situchan]

@rojiphil's proposal looks good to me. 🎀 👀 🎀 C+ reviewed

[melvin-bot[bot]]

Triggered auto assignment to @AndrewGable, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[laurenreidexpensify]

@AndrewGable bump to assign if you agree with proposal

[melvin-bot[bot]]

@AndrewGable @laurenreidexpensify @situchan this issue is now 4 weeks old and preventing us from maintaining WAQ, can you:

• Decide whether any proposals currently meet our guidelines and can be approved as-is today
• If no proposals meet that standard, please take this issue internal and treat it as one of your highest priorities
• If you have any questions, don't hesitate to start a discussion in #expensify-open-source

Thanks!

[melvin-bot[bot]]

Current assignee @situchan is eligible for the Internal assigner, not assigning anyone new.

[melvin-bot[bot]]

@AndrewGable, @laurenreidexpensify, @situchan Eep! 4 days overdue now. Issues have feelings too...

[laurenreidexpensify]

@AndrewGable bump ^ ^

[rojiphil]

The automatic offers did not get generated here. Looks like the default flow got disrupted somewhere. cc @laurenreidexpensify

[rojiphil]

@situchan PR is up for review.

[melvin-bot[bot]]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot[bot]]

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.4.45-6 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

• https://github.com/Expensify/App/pull/36531

If no regressions arise, payment will be issued on 2024-03-07. :confetti_ball:

For reference, here are some details about the assignees on this issue:

• @rojiphil requires payment (Needs manual offer from BZ)
• @situchan requires payment automatic offer (Contributor)

[melvin-bot[bot]]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [ ] [@situchan] The PR that introduced the bug has been identified. Link to the PR:
• [ ] [@situchan] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [ ] [@situchan] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [ ] [@situchan] Determine if we should create a regression test for this bug.
• [ ] [@situchan] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [ ] [@laurenreidexpensify] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[laurenreidexpensify]

Payment Summary:

• https://www.upwork.com/ab/applicants/1765716251259191296/job-details

C+ - @situchan $1000 offer sent in Upwork Contributor - @rojiphil $1000 offer sent in Upwork

[situchan]

This is not recent regression. The bug existed for long time. As this was reported by QA team, we can skip regression test.

[situchan]

@laurenreidexpensify I accepted new offer. You can close old contract

[rojiphil]

Contributor - @rojiphil $1000 offer sent in Upwork

@laurenreidexpensify I accepted the new offer. Thanks.

[laurenreidexpensify]

Payment Summary:

upwork.com/ab/applicants/1765716251259191296/job-details C+ - @situchan $1000 paid in Upwork Contributor - @rojiphil $1000 paid in Upwork