Closed sleepymalc closed 2 months ago
console.ts:137 [Extension Host] 5/16/2024, 6:32:01 PM [push] update "/Test/Figures/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 6:32:24 PM [pull] delete "/Test/Figures/Diploma.pdf" console.ts:137 [Extension Host] 5/16/2024, 6:32:29 PM [pull] delete "/Test/Figures/Diploma (Copy).pdf" console.ts:137 [Extension Host] 5/16/2024, 6:32:34 PM [pull] delete "/Test/Figures/test.pdf"
If the attached log is complete, it seems weird that server want to apply(pull) delete action, while there is no push action before.
Can you see what the time when the deleted files were pushed to the server?
Hmm, I'm copy-pasting random files (in this case my Diploma lol) around and decide to make a test.pdf
afterwards. Seems like some lagged action is performed after I switched to test.pdf
.
@iamhyc Please see the updated log:
console.ts:137 [Extension Host] 5/16/2024, 7:09:08 PM [push] update "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:09 PM [pull] update "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:12 PM [push] update "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:13 PM [pull] delete "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:13 PM [pull] update "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:14 PM [pull] delete "/test.pdf" console.ts:137 [Extension Host] 5/16/2024, 7:09:14 PM [pull] update "/test.pdf"
What I did: Paste a test.pdf
. Then wait.
I see. I will try to figure out why and fix it later.
Hi @sleepymalc , please help test the temporary extension here:
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9116597812
If there is no more problem, I will have the changes included in the next release.
It seems like there are some bugs. This is my first try on my current project:
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/f59e36ef-ceae-4905-90f5-aafd3124a6d7
What I did is just: copy those three files, paste them to the folder. The log is attached.
well, that is weird. I cannot reproduce the situation in my example project.
I tried (in the local folder, not in vscode), and nothing wrong happened.
The attached log is also weird. Anyway, I have tried to re-generate the extension, please try again and start with a small project firstly. https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9116597812
Okay, for a more complete demo, I record everything I did.
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/c252e896-6866-4bd9-8acd-3879aa08dafb
Sorry for the quality but I need to compress it in order to upload it here. The log is also attached.
Yes, I think you did nothing wrong, but I just cannot reproduce it.
I will try to find someone else to test for us.
From my knowledge of the workflow I use for auto-exporting these figures, a potential reason is that the workflow relies on fswatch
, which will monitor the file system. Not sure whether this is the issue.
From my knowledge of the workflow I use for auto-exporting these figures, a potential reason is that the workflow relies on
fswatch
, which will monitor the file system. Not sure whether this is the issue.
The sync also relies on vscode fswatch to monitor the changes, but I am not sure if they interfere with each other.
At least the test on Windows with local files copy/move works correctly now .
I think sometimes it works fine, but sometimes it doesn't. The interference between fswatch
can be the problem then.
I think sometimes it works fine, but sometimes it doesn't. The interference between
fswatch
can be the problem then.
if that is the case, please let me know how to reproduce it.
And I wonder if the workflow causes extra "delete action" when update the file. maybe the delete action is not handled properly now. I will take a look ata it later.
I have some new observations: basically whenever I update my figures, my workflow will export them once again, causing some over-written. And I think this will trigger some delete commands (base on the log)
While I'm not sure whether this is really the cause, but if an overwrite actually triggers a delete, then it makes sense as this will cause interference issues between the file system on the Overleaf server and the local file system.
I'm really not sure how to reproduce this, but here's one more observation: if you see my previous demo, this warning pops up:
I found out that this only happens when the file system on the Overleaf server thinks the pasted files already exists, i.e., I'm pasting a redundant file (in Overleaf server's perspective). So I guess some interference and mis-sync is happening.
More demo (@iamhyc maybe this helps):
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/9a88f239-b7de-41f0-93ef-d491e5f59a3c
What I did:
.svg
is added, which recompile the figures again and do an auto-exportxxx.pdf
(which the local file system doesn't have, but the Overleaf file system thinks it's already there), an new error is thrown, and the local file system deletes that xxx.pdf
xxx.pdf
from the Overleaf file system and do 4. again, nothing happens, everything works as expected.Note that 5. is different from 1. since in 1., the workflow do a recompilation (to make sure everything is up-to-date) and then auto-export them to over-write the old files. This doesn't happen for 5. as pasting xxx.pdf
won't trigger the workflow.
From my side, to reproduce the issue, the best way is just to re-paste an existing xxx.svg
. This will looks like this:
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/3b960570-7b5b-41c0-9b90-8ab9b8c4ed11
The first try is fine. But when I paste the xxx.svg
again to cause a re-compilation, things go wrong.
@sleepymalc Thank you. The demos are very clear. I find it interesting that the "pdf file delete" is initiated from overleaf server firstly.
I have update the temporary extension with file hash information included. Please help test and attach the log. We need more information to figure out why.
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9123020301
Okay, this is the new demo:
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/1bac7c33-47bb-43d4-bcaf-f17405ff4dc5
With the log attached:
Ok, I get the point. We should blame the overleaf server.
When overwrite existing file on overleaf server, it will trigger "delete-then-update" action instead of only update.
The denoised log are attached for your interest.
Hmm, in this case, is this fixable on the extension side? I would imagine if the file syncing is with some delay, then maybe the interference can be avoided? I don't know just a guess.
Hmm, in this case, is this fixable on the extension side? I would imagine if sync file updates with some delay maybe the interference can be avoided? I don't know just a guess.
Yes, it can be fixed and I just find some bugs in the previous implementation. I need some time to fix it.
Hi @sleepymalc , it now at least working on my side again, though with some error message.
Please try with the temporary extension here:
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9148809555
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9148904531 (this also solves #140)
This is my test demo:
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/92ad28eb-4235-450b-adc8-638e218f36fd
After initial pasting it fails already. Not sure whether it's because it's xxx.pdf
(but previously at least the first pasting is fine even if there's a xxx.pdf
).
Here's the log:
well, I believe there is something wrong with "vfs" (which connects to overleaf via websocket), but not local replica.
You can see there is no "delete action" push to server, but "vfs" detects the file deletion and try to sync to local replica.
I will check it later, but I think maybe we can merge the fixes into mainline for now, if there is nothing wrong except for the weird pdf file.
Actually I don't think it's xxx.pdf
's fault. I tried other pdf files but cannot recreate this issue: instead, this only happens when my workflow is exporting xxx.pdf
and xxx.pdf_tex
.
The workflow is based on Inkscape. So, I think maybe the easiest way to reproduce this problem is to use Inkscape and do a manual export and see... Not sure how much work will this be on your side.
https://github.com/iamhyc/Overleaf-Workshop/assets/60427058/eec0df02-0392-4b41-b89a-fb946f801c48
For example, I first export manually, which is fine. But when I do this for the second time and overwrite, things went wrong.
ok, I can try your workflow on my side. Just let me know the script for exporting the pdf file.
Hmm let me clarify: the workflow is about using a drawing app called Inkscape. I believe that this workflow is too complicated to set up but essentially what it does is when it sees a svg, it opens Inkscape, import the svg into which, and export it to Portable Document Format (*.pdf) (as used in the above demo), which creates xxx.pdf
and xxx.pdf_tex
that LaTeX can then parse.
I found out that to recreate the problem, it enough to manually export some *.svg
as Portable Document Format (*.pdf). By doing it twice (and replace the original ones), the problem shows up. So I guess using Inkscape suffices.
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9153963106
@sleepymalc This one should work better. I have not done thorough test, let me know if there are any bugs.
Still need fix.
Yes, I think I still encounter some bugs like not syncing perfectly between local and Overleaf server. But sometimes it works and at least it won't randomly delete my files.
https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9158121249?pr=138
@sleepymalc Please try this one.
It seems like there's no error message showing up, but when I re-export figures, the Overleaf doesn't seem to see the update.
It seems like there's no error message showing up, but when I re-export figures, the Overleaf doesn't seem to see the update.
Find a stupid typo. The updated extension is here: https://github.com/iamhyc/Overleaf-Workshop/actions/runs/9158931477?pr=138
Ah great. By a quick test everything seems to work. I'll keep using this extensively and see whether this is stable.
Describe the bug
Firstly, I have a workflow that will automatically export LaTeX-compatible figure files created by Inkscape (including
xxx.svg
,xxx.pdf
,xxx.pdf_tex
). It seems like when using local environment with Overleaf-Workshop, since the extension is trying to not uploading all random files, whenever my workflow is working (i.e., exporting files to some directory under the project), after a few seconds, the extension will delete bothxxx.svg
andxxx.pdf
. I consider this as a bug since only files that is produced after compilation should be discarded by this extension (e.g.,xxx.bbl
).Expected behavior
Not deleting these files uploaded by the user.
How To Reproduce
I tried to copy and paste a
pdf
file without using VS Code, and after a second it's deleted. I guess this reproduces the problem.Environment
[Optional] Developer Logs
Additional Information
Here's other information that might help: basically the sync between the local and Overleaf server is inconsistent. I observe that when I past the
xxx.pdf
to my local project folder without using VS Code, it actually syncs to the Overleaf server (hence the online project). However, later when it's deleted from my local environment,xxx.pdf
on the Overleaf server is still there. If I close VS Code and re open this project (using local edit again), it'll download the "deleted"xxx.pdf
from Overleaf server.