Files larger than 40 KB cannot be loaded on Seafile web UI, and cannot sync to another Obsidian device

[chunjiw]

Hi, I want to say thank you! I try it and immediately want to switch away from livesync to this. So much cleaner than livesync! Also I would be able to do daily backup of Obsidian notes together with (already daily backed up) Seafile libraries.

I just find out that images in the vault may be corrupted when synced to Seafile, because Seafile web UI cannot load the image. Happens to both png and jpeg. Also, when sync the vault (first time) to an Android phone, I see an error

Sync failed: HTTP 500. Response: {}

all text files are synced no problem, but not image files.

[conql]

Hi @chunjiw,

Thank you for the feedback! I have a few questions and suggestions to help diagnose the issue further:

1. Reproducibility: Does this issue happen every time an image is uploaded, or does it occur randomly? Are all image uploads affected, or just specific ones? How large are the images you're trying to upload? How many images are you adding at once? Sometimes network errors can result in partial uploads, especially with larger files or bulk uploads.

2. File Integrity Check: To check and repair file corruption, please execute the following command in the seafile server:

   cd seafile-server-latest
   ./seaf-fsck.sh [library-id1] [library-id2] ...

   More information can be found in the Seafile manual.

3. Concurrency Policy: The upload/download concurrency policy in the current version is a bit aggressive and might cause errors during the first-time download. I am working on a better design to handle upload/download concurrency.

---

You could:

• Try repairing the file integrity.
• Clear the repository in the plugin settings and sync from the start again.

If the problem persists and you have some time, it would be helpful to turn on debug mode in the plugin settings and share the log printed in the console.

Here is a guide to open the Obsidian app console:

1. Open the Electron app.
2. Press Ctrl+Shift+I (or Cmd+Option+I on Mac) to open the Developer Tools.
3. Navigate to the "Console" tab to view and copy the logs.

Sorry for the inconvenience caused and thank you for your support.

[chunjiw]

Thank you for your detailed response! These questions I have answers:

• Reproducibility: All image files, not random, all sizes, from a few KB to a few MB. About around 10 images at most at once. The files all appears in Seafile UI, just cannot show it. I'm sure it is not network issue.

• Attempt to fix: Clear the repo and sync from the start, then appears error message mentioned above, and image files cannot be displayed. At this point, in the vault folder, the image file has zero size.

• Log:

  Sync started
  plugin:seafile:6687 Error: HTTP 500. Response: {}
  at Server.sendRequest (plugin:seafile:5594:13)
  at async Server.requestSeafHttp (plugin:seafile:5603:12)
  at async Server.getBlock (plugin:seafile:5941:18)
  at async SyncController.downloadFile (plugin:seafile:6277:23)
  at async SyncController.pull (plugin:seafile:6422:11)
  at async Promise.all (index 5)
  at async SyncController.pull (plugin:seafile:6389:7)
  at async SyncController.sync (plugin:seafile:6643:5)
  at async SyncController.syncCycle (plugin:seafile:6685:9)
  syncCycle @ plugin:seafile:6687
  await in syncCycle (async)
  startSync @ plugin:seafile:6671
  enableSync @ plugin:seafile:7426
  await in enableSync (async)
  eval @ plugin:seafile:6798
  plugin:seafile:6691 Sync: 80.0947265625 ms

Could it be Seafile API issue?

[chunjiw]

Some other things I see:

On Seafile UI, on the console, there is a GET 400 (Bad Request) when the UI tries to get image thumbnails. This leads me believe the image files are corrupted during syncing, so Seafile could not generate thumbnails. Also when I try to download image files from Seafile web UI, I always get "Failed to Download".

[conql]

Each file in Seafile is stored as a series of blocks. The HTTP 500 error could occur when some of the blocks are missing, which is often due to a failure during file upload.

Normally, this should not happen. The plugin only commits changes to the Seafile system after a successful upload. So this could be due to potential bugs. Unfortunately, without further information, I am unable to provide an immediate fix.

You could try deleting those images in Seafile and uploading them again. Alternatively, you can use seaf-fsck, as mentioned above, to automatically delete corrupted files.

Contact me if the issue persists or if there are any updates!

[chunjiw]

I've tried deleting and syncing again, never succeeded. So far this only affects image files. I'm so curious now, I want to get to the bottom of this.

May I ask whether you are using Seafile Community v11 or Seafile Pro v10 in your setup? I want to rule out whether it is my Seafile server setup.

[chunjiw]

I found more info:

1. This is not about image files, this is about file size. Any file that is bigger than 40 KB fails at first sync. I've tested pdf, mp4, various text file types
2. If a file is once in Seafile, even deleted, the same file would have no issue syncing. Even if the file is deleted in Seafile.

This is how I found out:

1. Attempt to sync a file (e.g. big.jpg) bigger than 40 KB, it fails, cannot download it from Seafile
2. Upload the same file (e.g. renamed to bigcopy.jpg) to Seafile using other methods (I was using REST API and curl)
3. Now both big.jpg and bigcopy.jpg are good, have thumbnail, and can be downloaded.

[chunjiw]

So now I have a workaround: only add big files via other ways. I will start using this on my main vault!

[aoout]

Maybe there could be another plugin that compresses all image files larger than 40kb to 40kb?

[aoout]

I tested it and found that when the file uploaded by the plugin is larger than 40kb, it cannot be viewed and downloaded on the web UI. However, if the file is larger than 40kb and uploaded manually, it can be successfully downloaded by the plugin. Therefore, the problem lies in the upload process.

[chunjiw]

Maybe there could be another plugin that compresses all image files larger than 40kb to 40kb?

When I tested this issue, I had a new vault with only this plugin. So no, there's not another plugin messing it up.

[conql]

Sorry for the late response; I've been very busy lately.

The issue might be related to changes in the Seafile internal protocol. I was using version 9.0.4 during development.

I should be free to address this in about a week. If anyone can help in the meantime, feel free to submit a pull request. Code-related contributions are welcome.

Feel free to share more info, details, or questions.

Thanks for your understanding.