search

simulot / immich-go

An alternative to the immich-CLI command that doesn't depend on nodejs installation. It tries its best for importing google photos takeout archives.
GNU Affero General Public License v3.0
1.19k stars 35 forks source link

Random interruption of imports. No errors yielded #262

Closed LBlend closed 1 week ago

LBlend commented 1 month ago

I'm trying to import my takeout from google photos, of which there are four 50 GB files and one smaller one. However when running this command on my x86-64 linux machine.

./immich-go -server=MYSERVER -key=MYKEY upload -create-albums -google-photos ~/Downloads/takeout-*.zip

it seems to randomly stop after about 40 minutes or so without throwing any errors, leaving me with just a fraction of the files uploaded. Running the command multiple times yields the same results.

 INFO | 
Input analysis:
---------------------- | time="2024-05-26T05:41.33 CEST"
 INFO | scanned image file                      :   26795 | time="2024-05-26T05:41.33 CEST"
 INFO | scanned video file                      :    2724 | time="2024-05-26T05:41.33 CEST"
 INFO | scanned sidecar file                    :   29019 | time="2024-05-26T05:41.33 CEST"
 INFO | discarded file                          :    1199 | time="2024-05-26T05:41.33 CEST"
 INFO | unsupported file                        :       0 | time="2024-05-26T05:41.33 CEST"
 INFO | file duplicated in the input            :   12902 | time="2024-05-26T05:41.33 CEST"
 INFO | associated metadata file                :   29519 | time="2024-05-26T05:41.33 CEST"
 INFO | missing associated metadata file        :       0 | time="2024-05-26T05:41.33 CEST"
 INFO | 
Uploading:
---------- | time="2024-05-26T05:41.33 CEST"
 INFO | uploaded                                :    3652 | time="2024-05-26T05:41.33 CEST"
 INFO | server error                            :       0 | time="2024-05-26T05:41.33 CEST"
 INFO | file not selected                       :     382 | time="2024-05-26T05:41.33 CEST"
 INFO | server's asset upgraded with the input  :       0 | time="2024-05-26T05:41.33 CEST"
 INFO | server has same photo                   :       0 | time="2024-05-26T05:41.33 CEST"
 INFO | server has a better asset               :       0 | time="2024-05-26T05:41.33 CEST"

image

As you can see all the log files are nearly the exact same file size. Meaning they basically match in output

Any ideas as to what might be causing this? As I'm not getting any errors it's hard to tell.

commit f8e7b42b29cdae75be69afe2181dfc58b58cc674, built at 2024-05-20T13:44:20Z

simulot commented 1 month ago

The report shows 12902 duplicated files in the input.

Immich-go upload only one of them.

The high number of duplicates sounds strange. You can check the concerned files in the log, and verify in the input archive.

LBlend commented 1 week ago

Redownloading the takeouts as well as updating immich and immich-go to the latest version seems to have improved the situation though I'm not confident everything has been uploaded yet. Running it multiple times always yields 300 or so files uploaded. It's a weird issue