search

sboesen / remotely-sync

fork of remotely-save with security upgrades
Apache License 2.0
213 stars 8 forks source link

[Bug]: When synchronizing larger files,it will show synchronization interruption and report an error #36

Closed muhanstudio closed 1 year ago

muhanstudio commented 1 year ago

What happened?

When synchronizing larger files, such as PDF files over 50MB, it will show synchronization interruption and report an error:Error: 20231101202118-coowegz.pdf: Request failed, status 400 at eval (plugin:remotely-secure:112:51353) image

What OS are you using?

Windows, iOS (iPhone, iPad), Android

What remote cloud services are you using?

S3

Version of the plugin

lastest

Version of Obsidian

lastest

Using password or not

Ensure no sensitive information

muhanstudio commented 1 year ago

It is also accompanied by warnings:plugin:remotely-secure:76 Are you using a Stream of unknown length as the Body of a PutObject request? Consider using Upload instead from @aws-sdk/lib-storage. image And it can also lead to some incorrect output at startup: image image Thanks a lot to the author for redeveloping and maintaining this plugin, if these bugs can be fixed, the plugin can work better!

sboesen commented 1 year ago

Thank you for the details and appreciate you filing this issue! Sorry about the bug, will try to get this tested/fixed.

I have noticed that "Upload" warning as well, it occurs for regular file uploads so I'm not sure it's related to the error you're experiencing. Will investigate.

sboesen commented 1 year ago

@muhanstudio are you using an S3 provider with a file size limit, maybe configured? I just tested using storj.io and was able to sync a 53 megabyte test file without any errors.

I did experience a crash (entire obsidian window went white, dev tools disconnected) when trying to upload files > 200 megabytes, which is definitely an issue. I didn't get an error like you saw in the console, though - that looks like the remote server is rejecting the request, which I don't expect.

muhanstudio commented 1 year ago

Thanks for the speed, it seems that the file size over 50MB is not the real cause of the synchronization error, after I just checked the data stream, I found that the problem is the global agent of V2ray(a proxy tool), V2ray truncated the data. Even if I put the S3 domain name in the whitelist, this situation will still occur. After I directly shut down all agents or switch the proxy mode to PAC mode and add the S3 domain name to the whitelist, everything will return to normal, the synchronization tool will no longer report errors, and all files will be synchronized normally. I hope my experience can help others, and thank you for troubleshooting the problem.

sboesen commented 1 year ago

Thanks for confirming! I'll close this issue as it looks like you've identified the root cause of that console error, but we definitely have a large filesize issue which I'm tracking now in #42. Appreciate the bug report!

FEI352 commented 1 year ago

Thanks for the speed, it seems that the file size over 50MB is not the real cause of the synchronization error, after I just checked the data stream, I found that the problem is the global agent of V2ray(a proxy tool), V2ray truncated the data. Even if I put the S3 domain name in the whitelist, this situation will still occur. After I directly shut down all agents or switch the proxy mode to PAC mode and add the S3 domain name to the whitelist, everything will return to normal, the synchronization tool will no longer report errors, and all files will be synchronized normally. I hope my experience can help others, and thank you for troubleshooting the problem.

如果一开始存储的地方选择香港，这种情况会不会好点。