Open lanitochka17 opened 3 weeks ago
melvin-bot[bot]
commented
3 weeks ago Triggered auto assignment to @kevinksullivan (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.
NJ-2020
commented
3 weeks ago mWeb - Attachment - Opened offline attachment directed to conversation page on online.
When we are offline, we upload an attachment and click on it to view the details by clicking the attachment. When we return online, we invoke submitAndClose function because an attachment has been sent. Inside this function, we close the modal with setIsModalOpen(false) after sending the attachment.
We can add a new parameter to the URL or a parameter inside the function to indicate how the AttachmentModal was opened. For example, we can use ?openType=openDetail when the user opens the attachment by clicking on it, and ?openType=submit when the user submits the attachment. We can then conditionally close the modal based on the value of openType.
We can check if there is existing Onyx data for the attachment. If there is, it means the attachment has been submitted but not yet added to the backend. In this case, we can avoid invoking setIsModalOpen(false) and keep the modal open.
tsa321
commented
3 weeks ago Edited by proposal-police: This proposal was edited at 2024-08-28 14:38:24 UTC.
When opening a sent attachment while offline, after going online, the app navigates to a report.
When viewing an attachment image, the AttachmentCarousel is displayed. It uses logic to determine if the viewed image has been deleted then dismiss the modal. This is done by comparing the source (a URL parameter) with the available attachment properties in reportActions:
The lines detects if the viewed image is deleted by comparing the current source with the updated attachment properties retrieved from reportActions.
When the user is offline, the current source is a blob URL string. When the user goes online, this source is updated to a new path.
The current detection logic mistakenly identifies the image as deleted because the blob URL is no longer available, resulting in the dismissal of the modal.
We could add a check in the if clause to determine whether the current source is an image that will be uploaded, by verifying if the source starts with blob and if the new attachment has the same file name as the uploaded image.
The revised code could look like this, assuming a report action can have multiple images (i.e., an array):
if (initialPage === -1 && attachments.find(compareImage)) {
let isUploadedImage = false;
// maybe cheching for wheter the file is blob uri object is enough, but for more code completeness I am adding additional checks.
// If we want to checks whether the blob uri object exist we can use `FileUtils.readFileAsync`
if (source.startsWith('blob')) {
const uploadedAttachment = attachments.find(compareImage);
const possibleTargetUploads = targetAttachments.filter(attachment => {
return attachment.reportActionID === uploadedAttachment.reportActionID;
});
isUploadedImage = possibleTargetUploads.some(attachment => {
return attachment.file.name === uploadedAttachment.file.name;
});
}
if (!isUploadedImage) {
Navigation.dismissModal();
}
}Alternatively, we could check for pending actions such as deletion or addition, if this approach is more accurate, or simply verify whether the reportAction is a deleted message.
We can determine whether the report action has been deleted by checking the availability of currently viewed report attahcment's reportActionID in targetAttachment. Then set the search result to initialPage.
The code could be:
const currentAttachmentReportActionID = attachments.find(compareImage)?.reportActionID;
let initialPage;
if (currentAttachmentReportActionID) {
initialPage = targetAttachments.findIndex(attachment => attachment.reportActionID === currentAttachmentReportActionID);
} else {
initialPage = targetAttachments.findIndex(compareImage);
}Or :
let initialPage = targetAttachments.findIndex(compareImage);
const currentAttachmentReportActionID = attachments.find(compareImage)?.reportActionID;
const isReportActionExist = currentAttachmentReportActionID ? targetAttachments.find(attachment => attachment.reportActionID === currentAttachmentReportActionID) : initialPage !== -1;
// Dismiss the modal when deleting an attachment during its display in preview.
if (initialPage === -1 && attachments.find(compareImage)) {
if (!isReportActionExist) {
Navigation.dismissModal();
}
}Expanding my first solution which I mention to use pendingAction data:
We could add pendingAction data to the attachment. If the report action for the currently viewed attachment is pendingAction add, we should not dismiss the modal.
In this line:
We need to modify it to:
(Additionally, some modifications to the replace function are based on the bug I mentioned at the bottom of my proposal):
const html = ReportActionsUtils.getReportActionHtml(action).replace(/(<(?:(?=video )|(?=img )).+?)(\/*)>/gm, `$1 data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}" data-pending-action="${action?.pendingAction}"$2>`);Then, in this line and this line, we should add:
pendingAction: attribs['data-pending-action']In the AttachmentCarousel, we modify the lines near the dismiss modal to:
const currentlyViewedAttachment = attachments.find(compareImage);
if (initialPage === -1 && !!currentlyViewedAttachment) {
if (currentlyViewedAttachment?.pendingAction !== CONST.RED_BRICK_ROAD_PENDING_ACTION.ADD) {
Navigation.dismissModal();
}
}This is a simplified version of my initial solution to determine whether a file is stored temporarily by the browser or app. We can check if the source starts with "blob" or "file," or similar characters that indicate it is a file stored in user storage. The code could be:
if (initialPage === -1 && !!currentlyViewedAttachment) {
if (source.startsWith('blob') || source.startsWith('file')) {
return;
}
Navigation.dismissModal();
}
Also there is a bug in when user send a text with image attachment, the reportActionID data will be undefined. This in because:
If user send a text comment and an image, there is <br /> tag and the end tag of br that will be replaced. the more correct code could be:
const html = ReportActionsUtils.getReportActionHtml(action).replace(/(<img .+?)\/>/gm, `$1 data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}"/>`);To also fix for video:
const html = ReportActionsUtils.getReportActionHtml(action).replace(/(<(?:(?=video )|(?=img )).+?)(\/*)>/gm, `$1 data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}"$2>`);
daledah
commented
3 weeks ago Edited by proposal-police: This proposal was edited at 2024-08-29 02:10:22 UTC.
Upload and open attachment in offline, then going online user directed to conversation page
We have logic to close the attachment modal if User A is viewing an attachment and it gets deleted by User B: https://github.com/Expensify/App/blob/6697c267116daaa11c1a3c8a2c1009c90326451a/src/components/Attachments/AttachmentCarousel/index.tsx#L91-L93
In this logic, we check if the currently open attachment (identified by the source prop) was present in the previous list of attachments (identified by the attachments state) but no longer appears in the updated list (identified by the targetAttachments variable). If this condition is met, we call Navigation.dismissModal() to close the modal.
When we upload an image while offline and then reconnect online, the attachment source data switches from a blob value to the value returned by the backend, triggering the condition mentioned above to be true.
Since the scenario "User A is viewing an attachment and then it gets deleted by User B" only occurs with attachments that are not local files, we only need to update the condition accordingly: https://github.com/Expensify/App/blob/6697c267116daaa11c1a3c8a2c1009c90326451a/src/components/Attachments/AttachmentCarousel/index.tsx#L93
if(isLocalFile(source)){
return
}
Navigation.dismissModal();The same should be applied to native file.
Beside of using isLocalFile, we can make use of attribs[CONST.ATTACHMENT_OPTIMISTIC_SOURCE_ATTRIBUTE] by returning it in:
https://github.com/Expensify/App/blob/fc849dda35628b8c6c8d6edfa66dd31b257e8e7a/src/components/Attachments/AttachmentCarousel/extractAttachments.ts#L76
then we will use that data instead of the isLocalFile above to check if the attachment is local file or not. The similar logic is used in here:
https://github.com/Expensify/App/blob/fc849dda35628b8c6c8d6edfa66dd31b257e8e7a/src/pages/home/report/ContextMenu/ContextMenuActions.tsx#L572
daledah
commented
3 weeks ago melvin-bot[bot]
commented
2 weeks ago Job added to Upwork: https://www.upwork.com/jobs/~01e366edd029d4e402
melvin-bot[bot]
commented
2 weeks ago Triggered auto assignment to Contributor-plus team member for initial proposal review - @s77rt (External)
s77rt
commented
2 weeks ago @NJ-2020
When we return online, we invoke
submitAndClosefunction
This is not the case. The function is invoked as soon as you click the submit button (while offline)
s77rt
commented
2 weeks ago @tsa321 Your RCA is correct. However the suggested solution is a workaround. The correct approach would be to make the comparison function correctly identify that we are viewing an image that does exist.
s77rt
commented
2 weeks ago @daledah Your RCA is correct but same note as above regarding the solution. The attachment carousel should be able to tell that we are still viewing the same image even though the source did change. IOW initialPage should not be -1 because the image we are viewing does exist.
s77rt
commented
2 weeks ago It's worth mentioning that the targetAttachments values has a reportAction key which remains unchanged. We can make use of that.
daledah
commented
2 weeks ago s77rt
commented
2 weeks ago @daledah A file name is not a unique identifier and thus we cannot use it for lookup.
tsa321
commented
2 weeks ago daledah
commented
2 weeks ago @s77rt
A file name is not a unique identifier and thus we cannot use it for lookup.
It's worth mentioning that the targetAttachments values has a reportAction key which remains unchanged. We can make use of that.
Imagine user A sends a reportAction containing 3 images to user B, all linked to a single reportAction key.
User B opens the carousel to view the 2nd image.
Then, user A edits the reportAction and removes the 2nd image.
As a result, for user B, the carousel does not close.
s77rt
commented
2 weeks ago @tsa321 Thanks, ~looks good overall~
Edit: see https://github.com/Expensify/App/issues/48173#issuecomment-2323225431
s77rt
commented
2 weeks ago @daledah I'm not sure if multiple images in one report action is supported. So far I keep getting empty images, let me double check
s77rt
commented
2 weeks ago @daledah It turns out we can send multiple images in one report action (although it seems a bit broken; the report action id for the second image is undefined)
Thus we can't use the report action id for image identification either cc @tsa321
tsa321
commented
2 weeks ago on alternative solution 2 and 3.
Additionally, I have proposed a fix on the issue with incorrect reportActionID values (undefined) when a user sends an attachment with text or when an attachment is sent using markdown at the bottom of my proposal.
daledah
commented
2 weeks ago s77rt
commented
2 weeks ago @tsa321 The alternative solutions are still workarounds i.e. the correct image is still not being found and this is causing the component to have incorrect state (e.g. attachments is stale). One of the side effects is that on refresh we get the not found view, where we should have gotten the correct image.
s77rt
commented
2 weeks ago @daledah Same note ^ The proposed solution is a workaround. We should focus on the root cause that is the image is not being found and not on the image being dismissed. Fixing the former will by nature fix the later.
melvin-bot[bot]
commented
2 weeks ago @s77rt, @kevinksullivan Whoops! This issue is 2 days overdue. Let's get this updated quick!
s77rt
commented
2 weeks ago Still looking for proposals
melvin-bot[bot]
commented
1 week ago 📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸
dominictb
commented
1 week ago mWeb - Attachment - Opened offline attachment directed to conversation page on online
The condition: https://github.com/Expensify/App/blob/6697c267116daaa11c1a3c8a2c1009c90326451a/src/components/Attachments/AttachmentCarousel/index.tsx#L92 will be true when we change from offline to online because at that time, the current uri to display is local uri, so the initialPage: https://github.com/Expensify/App/blob/6697c267116daaa11c1a3c8a2c1009c90326451a/src/components/Attachments/AttachmentCarousel/index.tsx#L89 will be -1 and attachments.find(compareImage) is true.
In here and here, we should send an additional param to BE, like localUri = file.uri to BE.
Then, BE will attach that localUri data to the reportAction's html such as:
<img src="" data-expensify-source="" data-name="" localUri="blob:https://dev.new.expensify.com:8082/8c5b5171-0e8e-428f-8fe6-94bd5aaxd1352" />The, FE will return that localUri in here.
Finally, update this to:
const compareImage = useCallback((attachment: Attachment) => attachment.source === source || attachment.localUri === source, [source]);@dominictb Thanks for the proposal. The RCA makes sense but the solution does not feel right and having the BE deal with FE issues is not really something we want to implement.
melvin-bot[bot]
commented
1 week ago @s77rt @kevinksullivan this issue was created 2 weeks ago. Are we close to approving a proposal? If not, what's blocking us from getting this issue assigned? Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!
s77rt
commented
1 week ago Still no clear solution yet
kevinksullivan
commented
1 week ago Still waiting on proposals. Keeping price as I don't think this is worth expediting among other initiatives. I'm looping in another BZ as well since I'm going OOO.
melvin-bot[bot]
commented
1 week ago Triggered auto assignment to @mallenexpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.
melvin-bot[bot]
commented
6 days ago 📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸
wildan-m
commented
3 days ago Edited by proposal-police: This proposal was edited at 2024-09-17 20:59:02 UTC.
Opened offline attachment redirected to conversation page when online.
This check will be true when online, as the local blob url will be replaced with the remote url and initialPage will be -1
Introduce a new reportAction field optimisticSrc to store the optimistic (unuploaded blob) URL.
Fill the value when we buildOptimisticAddCommentReportAction
Extract value when we create targetAttachments:
...
hasBeenFlagged: attribs['data-flagged'] === 'true',
optimisticSrc: attribs['data-optimistic-src'],
});
....Pass the value to the generated HTML. If the text contain images, the uploaded file will be at the last position of closing tag />
....
const optimisticSrc = (action as OptimisticAddCommentReportAction).optimisticSrc;
const optimisticSrcAttribute = optimisticSrc ? `${CONST.ATTACHMENT_OPTIMISTIC_SOURCE_ATTRIBUTE}="${optimisticSrc}"` : '';
const html = ReportActionsUtils.getReportActionHtml(action)
.replace('/>', `data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}" />`)
.replace(/\/>(?!.*\/>)/, ` ${optimisticSrcAttribute}/>`);
htmlParser.write(html);
});....When report action edited, all of the images already uploaded, we have to remove optimisticSrc from onyx since the position of image might be changed.
src/libs/actions/Report.ts > editReportComment
....
lastModified: DateUtils.getDBTime(),
optimisticSrc: null,
},
....Modify compareImage in AttachmentCarousel for all platforms:
const compareImage = useCallback((attachment: Attachment) => attachment.source === source || (!!attachment.optimisticSrc && attachment.optimisticSrc === source), [source]);Then make isUploading value more accurate by identify it's source url prefix, :
Apply
const isUploading = CONST.ATTACHMENT_LOCAL_URL_PREFIX.some((prefix) => source.startsWith(prefix));In ContextMenuActions.tsx and AttachmentCommentFragment.tsx
We can store the reportActionID-optimisticSrc in a cache:
src/components/Attachments/AttachmentCarousel/extractAttachments.ts
const reportActionOptimisticSrcCache = new Map<string, string>()
function extractAttachments(
.....Use existing optimisticSrc
const htmlParser = new HtmlParser({
onopentag: (name, attribs) => {
const optimisticSrc = attribs[CONST.ATTACHMENT_OPTIMISTIC_SOURCE_ATTRIBUTE];
const reportActionID = attribs['data-id'];
if (optimisticSrc) {
reportActionOptimisticSrcCache.set(reportActionID, optimisticSrc);
} Pass the value to attachments.unshift({
attachments.unshift({
.......
optimisticSrc: reportActionOptimisticSrcCache.get(reportActionID),
});Ensure the last attachment have data-id (for case multiple images)
const html = ReportActionsUtils.getReportActionHtml(action)
.replace('/>', `data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}"/>`)
.replace(/\/>(?!.*\/>)/, `data-id="${action.reportActionID}"/>`); Modify compareImage function
src/components/Attachments/AttachmentCarousel
const compareImage = useCallback((attachment: Attachment) => attachment.source === source || (!!attachment.optimisticSrc && attachment.optimisticSrc === source), [we can clear the cache when reportID change
Generate sequence id and ensure report action id filled. report action ID always exists on the first open tag src/components/Attachments/AttachmentCarousel/extractAttachments.ts
onopentag: (name, attribs) => {
if (!firstDataID) {
firstDataID = attribs['data-id'];
}
if(reportActionIsEdited === undefined) {
reportActionIsEdited = attribs['data-is-edited'] === 'true';
} attachments.unshift({
reportActionID: firstDataID,
...........
sequenceID: sequenceID++,
reportActionIsEdited: reportActionIsEdited,
});Add isEdited to know if the reportAction edited or not, this will ensure no false positive case.
reset the sequenceId, reportActionId and reportActionIsEdited on each htmlParser.write
const isEdited = Array.isArray(action.message) && action.message[0] ? (action.message[0] as Message).isEdited : false;
const html = ReportActionsUtils.getReportActionHtml(action).replace('/>', `data-flagged="${hasBeenFlagged}" data-id="${action.reportActionID}" data-is-edited="${isEdited}"/>`);
//reset sequenceID and firstDataID for each action
sequenceID = 0;
firstDataID = '';
reportActionIsEdited = undefined;
htmlParser.write(html);capture prevTargetAttachmentsRef at the end of useEffect
src/components/Attachments/AttachmentCarousel/index.tsx
prevTargetAttachmentsRef.current = targetAttachments;modify compareImage
const compareImage = useCallback((attachment: Attachment) => {
const prevOriginalAttachment = prevTargetAttachmentsRef.current.find((prevAttachment) => !attachment.reportActionIsEdited && prevAttachment.reportActionID === attachment.reportActionID && prevAttachment.sequenceID === attachment.sequenceID);
return attachment.source === source || prevOriginalAttachment?.source === source;
}, [source]);@wildan-m Thanks for the proposal. The RCA is correct. The suggested solution looks good to me but can we use data-optimistic-src attribute? That attribute is already being added to the report action html message.
daledah
commented
2 days ago s77rt
commented
2 days ago @daledah I don't see why it wouldn't, each attachment would have its unique optimistic source
wildan-m
commented
2 days ago @s77rt That works but the suggested changes need to be kept to preserve the optimisticSrc even if the main source URL changes.
daledah
commented
2 days ago I don't see why it wouldn't, each attachment would have its unique optimistic source
I saw in that solution, we store the optimisticSrc in report action. I think it only works in case we have only 1 attachment in action. I wonder how we can store the optimisticSrc data in case the action has more than 1 attachment?
wildan-m
commented
2 days ago I think optimistic case only support single image upload at the moment, multiple image by markdown is not previewed when offline
wildan-m
commented
2 days ago s77rt
commented
2 days ago @wildan-m Why are you still passing more data into buildOptimisticAddCommentReportAction? We already have what we need in attachmentHtml
https://github.com/Expensify/App/blob/14b99ca0a12e9686818bc3e937f091199de69750/src/libs/ReportUtils.ts#L4134
s77rt
commented
2 days ago @daledah The extractAttachments logic will extract attachments from a each report action independently based on its html message, if it contains 2 attachments, then two objects will be returned.
wildan-m
commented
2 days ago Why are you still passing more data into buildOptimisticAddCommentReportAction? We already have what we need in attachmentHtml
@s77rt that generated html will be overriden from server, so we need to store optimistic source differently and not removing it even if the main source uri replaced
daledah
commented
2 days ago if it contains 2 attachments, then two objects will be returned.
Yes, I agree. However, I'm curious—if the action contains two attachments and we return the two objects you mentioned via https://github.com/Expensify/App/blob/14b99ca0a12e9686818bc3e937f091199de69750/src/components/Attachments/AttachmentCarousel/extractAttachments.ts#L76
optimisticSrc: attribs['data-optimistic-src'],, how will their optimisticSrc data appear? The optimisticSrc in both objects will be the same, as they originate from a single source, which is:
const optimisticSrc = (action as OptimisticAddCommentReportAction).optimisticSrc;
const optimisticSrcAttribute = optimisticSrc ? `data-optimistic-src="${optimisticSrc}"` : '';right?
wildan-m
commented
2 days ago https://github.com/Expensify/App/issues/48173#issuecomment-2357165519
IMO currently optimistic preview only support for single image, then using optimisticSrc would be enough, but if the optimistic preview will support multiple images, we can modify the var to array e.g. optimisticSources and manipulate them in order
s77rt
commented
1 day ago @wildan-m
that generated html will be overriden from server, so we need to store optimistic source differently
Ah I see. I'd really prefer to not save extra data in Onyx especially since it's data that we already have. In the attachment carousel the attachments array is updated through useEffect (side effect), at that time we can access both the old attachment (which contains the optimistic source) and the new attachment (from the server). Maybe we can find a way to link the two? (e.g. a way to know that this old attachment has become this new attachment)
daledah
commented
1 day ago IMO currently optimistic preview only support for single image
I mean the case:
Open any report.
Type message:
Click "Add attachment" in "+" button beside the composer > Choose any image > Click "Send".
Now, the message containing 3 images is sent.
In this case, the message returns by BE is:
[
{
"html": "<img src=\"https://img.freepik.com/free-photo/autumn-tree-forest-leaves-bright-yellow-generative-ai_188544-12668.jpg\" alt=\"image1.jpeg\" /><br /><br /><img src=\"https://img.freepik.com/free-photo/photorealistic-view-tree-nature-with-branches-trunk_23-2151478039.jpg\" alt=\"image2.jpeg\" /><br /><br /><img src=\"https://www.expensify.com/chat-attachments/7927659170597592909/w_6007da3bf757b9489e3b34fa1fbaff902784884f.jpg.1024.jpg\" data-expensify-source=\"https://www.expensify.com/chat-attachments/7927659170597592909/w_6007da3bf757b9489e3b34fa1fbaff902784884f.jpg\" data-name=\"360_F_182011806_mxcDzt9ckBYbGpxAne8o73DbyDHpXOe9.jpg\" data-expensify-height=\"360\" data-expensify-width=\"360\" />",
}
]and only optimisticSrc value mentioned in your proposal is not enough to differentiate between these images.
wildan-m
commented
1 day ago Maybe we can find a way to link the two? (e.g. a way to know that this old attachment has become this new attachment)
@s77rt I doubt that's possible, there is no unique identifier for each attachment
wildan-m
commented
1 day ago Thanks for the case @daledah, if we have multiple images, then uploaded image from file (not from text) will always at the last position, so we only need to add optimisticSrc attribute for the last image.
When report action edited, all of the images already uploaded, and no feature to directly upload the image, we have to remove optimisticSrc from onyx since the position of image might be changed / removed.
src/libs/actions/Report.ts > editReportComment
....
lastModified: DateUtils.getDBTime(),
optimisticSrc: null,
},
....
If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!
Version Number: 9.0.25 Reproducible in staging?: Y Reproducible in production?: Y If this was caught during regression testing, add the test name, ID and link from TestRail: N/A Issue reported by: Applause - Internal Team
Action Performed:
Expected Result:
Upload and open attachment in offline, then going online user must stay in same attachment page
Actual Result:
Upload and open attachment in offline, then going online user directed to conversation page
Workaround:
Unknown
Platforms:
Which of our officially supported platforms is this issue occurring on?
Screenshots/Videos
Add any screenshot/video evidence
https://github.com/user-attachments/assets/083f0cf4-a3a8-4dad-864d-7bb3184eda93
View all open jobs on GitHub
Upwork Automation - Do Not Edit
Issue Owner
Current Issue Owner: @s77rt