Some files are silently ignored

[Guekka]

Command ran: immich-go -debug -log-level DEBUG -server <URL> -key <KEY> upload 2019/2019-04/VID_20190419_194331.mp4

The file is not uploaded to the server but the log do not show it. Log:

DEBUG | Connection to the server <URL> | time="2024-06-02T03:21.29 BST"
DEBUG | Server status: OK | time="2024-06-02T03:21.29 BST"
DEBUG | Connected, user: <USER> | time="2024-06-02T03:21.29 BST"
DEBUG | Browsing folder(s)... | time="2024-06-02T03:21.29 BST"
DEBUG | scanned video file | file=VID_20190419_194331.mp4 time="2024-06-02T03:21.33 BST"
DEBUG | Managing albums | time="2024-06-02T03:21.43 BST"
DEBUG | 
Input analysis:
---------------------- | time="2024-06-02T03:21.43 BST"
DEBUG | scanned image file                      :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | scanned video file                      :       1 | time="2024-06-02T03:21.43 BST"
DEBUG | scanned sidecar file                    :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | discarded file                          :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | unsupported file                        :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | file duplicated in the input            :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | associated metadata file                :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | missing associated metadata file        :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | 
Uploading:
---------- | time="2024-06-02T03:21.43 BST"
DEBUG | uploaded                                :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | server error                            :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | file not selected                       :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | server's asset upgraded with the input  :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | server has same photo                   :       0 | time="2024-06-02T03:21.43 BST"
DEBUG | server has a better asset               :       0 | time="2024-06-02T03:21.43 BST"

The file is quite big, about 400mb.

[simulot]

Have you used the 0.16.0 version for this test?

I have fixed an error of this kind. A timeout error during the upload wasn't correctly handled and the error wasn't reported. I have fixed the report and extended the timeout value.

Could you check it?

[Guekka]

Have you used the 0.16.0 version for this test?

I have fixed an error of this kind. A timeout error during the upload wasn't correctly handled and the error wasn't reported. I have fixed the report and extended the timeout value.

Could you check it?

Oh, the release was 5 hours ago. I should have checked first, sorry. Going to try

[Guekka]

I can confirm the file is uploaded correctly with 0.16.0, thank you

[zdimension]

In case someone imported many folders using < 0.16, here is a small script that will detect files that have been skipped:

import os
import re
from pathlib import Path

path = Path(r"D:\Téléchargements\logs\immich-go") # put your log path here

# for each .log file in path
for file in [x for x in os.listdir(path) if os.path.isfile(path / x) and x.endswith(".log")]:
    with open(path / file, "r", encoding="utf-8") as f:
        content = f.read()
        if "scanned image file" not in content:
            continue

        scanned_images = set()

        for scanned in re.finditer(r'scanned (?:image|video) file \| file="([^"]+)"', content):
            assert scanned.group(1) not in scanned_images, scanned
            scanned_images.add(scanned.group(1))
        old = len(scanned_images)
        for uploaded in re.finditer(r'(?:uploaded|file duplicated in the input|server has same (?:asset|photo)) \| file="([^"]+)"', content):
            scanned_images.remove(uploaded.group(1))

        missing_meta = set()

        for uploaded in re.finditer(r'missing associated metadata file \| file="([^"]+)"', content):
            missing_meta.add(uploaded.group(1))
            scanned_images.remove(uploaded.group(1))

        if not missing_meta and not scanned_images:
            continue

        print(file, old, "remaining=", scanned_images, "missing meta=", missing_meta)
        print()

Simply relaunching immich-go is a fix if you have only one folder but in my case I imported tens of different folders so reimporting everything would have been awful

[simulot]

Thanks @zdimension, I hope the 0.16 fixes the problem. Let me know