mp4-files do not get imported

[hjjg]

Hi there,

thank you for the great tool!

I am using the latest version of immich (1.87.0) and the latest version of immich-go (downloaded today).

I just installed everything and directly started to import various directories, but then I noticed, that each and every mp4-file leads to this message: Uploading "DJI_0515.MP4"...already exists on the server

That happened to almost all videos (but I haven't checked them all). What does work is .mov. Do you have any idea on how to debug this?

Thanks in advance!

[simulot]

thank you for the great tool!

Thanks for your nice comment I'm a bit surprised... mp4 are very common.

Uploading "DJI_0515.MP4"...already exists on the server

This message is due to the server response after uploading the file. It seams the server knows this file already. Could you try to search it using by giving m:DJI_0515.MP4 in the search bar?

[hjjg]

Uploading "DJI_0515.MP4"...already exists on the server

This message is due to the server response after uploading the file. It seams the server knows this file already. Could you try to search it using by giving m:DJI_0515.MP4 in the search bar?

I did so, the file is not there. Crazy enough, I uploaded the file via the Webinterface after that and I get the same (wrong) message that the file is a duplicate (even though I thought I tested this before with another Video...).

Seems that your tool is just fine, so I stick with the nice comment and forget about the rest of my message :)

Thank you very much!

[simulot]

So the explanation is simple: Immich store the SHA1 hash in the database of each files. the hash depend on the content of the file. When a new file is uploaded, its hash is calculated and compared to the hashes already present in the database. If the new file's hash is present, immich reject it as a duplicate.

My guess is that you have already this file in immich, but with another name.

[hjjg]

I did further testing and I am not sure what to believe now - importing with immich-go there are plenty of videos that do not get imported. Tested it again and uploaded some files with the webinterface and they did not get marked as duplicate...

[hjjg]

This is one of the specific tests. I made sure that there is not a single video in my timeline for 2012/12, then I started the import:

immich-go -server http://10.110.0.4:2283 -key XXX upload *.mp4

immich-go 0.8.4, commit edc40a3188830af525aac577c532764cee03d279, built at 2023-11-19T15:17:40Z Server status: OK Ask for server's assets... 18056 asset(s) received Browsing folder(s)...Done. Uploading "2012-12-12_14-05-30_VID_20121212_150221.mp4"...already exists on the server Uploading "2012-12-12_150820.mp4"...already exists on the server Uploading "2012-12-15_185905.mp4"...already exists on the server Uploading "2012-12-17_110238.mp4"...already exists on the server Uploading "2012-12-22_174629.mp4"...already exists on the server Uploading "2012-12-24_190713.mp4"...already exists on the server Uploading "2012-12-27_105008.mp4"...already exists on the server Uploading "2012-12-08_195812.mp4"...already exists on the server Managing albums 8 media scanned, 0 uploaded. Done.

When searching for the filenames: nothing. Uploading via Webinterface -> duplicates.

Is there any way to add debugging output?

[simulot]

Immish just gives back the internal ID of the file it is already knows.

Thinking loud: I have received the the list all known file, with ID and name.... I'll report this information on the console.

[codokie]

I seem to have the same issue. Only one out of hundreds of .mp4 files was uploaded to a fresh Immich installation using the latest version of immich-go, while all .jpg, .png and .mov files were uploaded successfully. My guess is that all subsequent .mp4 files somehow got the same ID as that of the first uploaded .mp4 file. Is there any way to force an upload with duplicates? Edit: I think it may have to do something with the breaking changes following v1.88 release (?)

[hjjg]

Hi @codokie NICE, I am not alone 👍

Recent observation: If I create a new video on my phone it gets imported with no problem.

What I suspect: as soon as I try to upload a file with immich-go it is somehow remembered by immich and the rejected on any other attempt, even if I use the web client. If I try to add a video that has not been seen by immich-go it works with the web client.

EDIT: Same issue with 1.88.2 (was on 1.87.0).

[RareKiwi]

Having the same issue on all my .mp4 on the first import of an empty 1.88.2 immich

[codokie]

It seems that something has definitely changed under the hood in the last few versions of Immich, even the immich-cli now throws errors when using the upload command for any kind of media. That said, I was able to upload the .mp4 files rejected by immich-go with the web client, they were not rejected in my instance. @hjjg

[hjjg]

I have updated the issue in the immich project.

[simulot]

Have you applied all changes on the docker compose file? I had some errors too before I did the change.

[hjjg]

Sure thing, I am on the newest version and removed the lines for the two docker containers that have been consolidated. But I did get this behaviour with 1.87.0, too.

If you look at my bug report on the immich project you can see that there is indeed a database entry for a file that gets rejected as a duplicate, but this entry seems to be incomplete.

For now I suspect that immich-go creates the videos partially (if that is possible using the API) or the API has problems processing the file and too quickly responds with a positive message that the file has been uploaded successfully.