search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
51.94k stars 2.75k forks source link

[BUG] App upload progress is reset when the screen goes off #747

Closed oddlama closed 1 year ago

oddlama commented 2 years ago

Describe the bug I have about 3000 assets on my phone that need to be synced (some of those should already exist, but apparently the app doesn't check whether certain checksums already exist). I set my screen-off time to the maximum possible value on my Huawei phone which is 10 minutes. I manually start a backup and everything seems to working fine.

The problem is when I miss the opportunity to stop my phone from locking the screen, the upload progress is stopped probably because the phone goes to sleep. After unlocking the phone, the remainder is correctly displayed wherever the process stopped (last time at around 1000 remaining assets). When I now press the button "Start backup" again, the remainder is immediately reset and all 3000 assets are uploaded again.

It sounds to me like the app isn't saving it's state regularly when it changes, which causes it to be reset when the app lifecycle is stopped and resumed. I would even argue that ideally the client should send the server a list of checksums and the server should return only the indices in that list which the server doesn't already know. This way the state of already known images could be updated quickly by the client and only the files that actually need to be uploaded would actually be uploaded. Currently I had to exclude several gigabytes of videos to at least get all the images backed up (the videos just take too long to reupload on each try).

EDIT: I now managed to complete all uploads, but still the progress hasn't been saved after I locked my phone :( Apparently the state isn't even saved on completion, I'm not sure what to do to trigger a save.

To Reproduce Steps to reproduce the behavior:

  1. Have a preexisting folder on your phone with some assets
  2. Select them for backup and start the backup process
  3. Lock the phone mid-backup and wait some minutes
  4. Unlock and resume backup to see it will reset to before starting the initial backup

Expected behavior The app's state should be saved regularly

System

alextran1502 commented 2 years ago

Firstly, sorry for the inconvenience. This error seems to be specific to the phone model as I cannot reproduce it on my Samsung S9. My steps are as follow, let me know if they are the same as yours

  1. Sign in
  2. Go to backup screen
  3. Choose Recent (All) album/folder
  4. Press Start Backup
  5. After a few assets had been uploaded, turn off the screen.
  6. Unlock and resume the app.
  7. I saw the number of the Backup and Remainder have changed accordingly
  8. Press Start Backup.
  9. The number of Backup continue to count up while the Remainder counts down as expected.

Question

What are the state of the following buttons on your phone? image

More context

The state of the count is updated when

  1. Entering the backup page, the app will fetch the data from the server to compare which assets have been backup and which are the left over.
  2. When you press the Start backup button, it will perform the similar action above.
  3. The local states include the count up and count down change which should always be the when get recalculated with the server's states.

Hint: You can turn on background service and the app will perform the backup in the background so that you don't have to leave the phone on.

oddlama commented 2 years ago

Firstly, sorry for the inconvenience.

Oh no worries! I'm just trying to help, I don't strictly depend on any of this :) I knew what I was getting into when you said it isn't production ready.

This error seems to be specific to the phone model as I cannot reproduce it on my Samsung S9. My steps are as follow, let me know if they are the same as yours

1. Sign in

2. Go to backup screen

Is it normal that it takes ~3 seconds for the app to change from "Selected: none" to the actual selected assets when I go here?

3. Choose `Recent (All)` album/folder

I can't do that unforunately, I don't want all images on my phone to be uploaded as that would include whatsapp images and other things that I don't want. Instead I have chosen two of the shown folders (Camera, Snapchat).

4. Press `Start Backup`

At this point I get Total 2987, Backup 235, Remainder 2752. That is also the state to where I'm reset all the time.

5. After a few assets had been uploaded, turn off the screen.

6. Unlock and resume the app.

7. I saw the number of the `Backup` and `Remainder` have changed accordingly

Same here, the displayed state is still correct until I press start again.

8. Press `Start Backup`.

9. The number of `Backup` continue to count up while the `Remainder` counts down as expected.

It takes 3 seconds where it calculates something, then the count is reset to 235 and the process begins anew.

Question

I have both of those off at the moment (i though it might be related to the problem), but just tested it again with both of the buttons turned on. The same thing happens.

More context

The state of the count is updated when

1. Entering the backup page, the app will fetch the data from the server to compare which assets have been backup and which are the left over.

How is the comparison done currently? With checksums or file names? I did complete a full backup once on my phone today by continuously tapping the screen, and it doesn't detect that these have already been uploaded.

2. When you press the `Start backup` button, it will perform the similar action above.

Interesting... can I do something to help here? Do you need logcat logs?

3. The local states include the count up and count down change which **should** always be the when get recalculated with the server's states.

Hint: You can turn on background service and the app will perform the backup in the background so that you don't have to leave the phone on.

That's what I did initially but when i noticed that it was reset periodically i tried to solve it by doing manual backups. I also followed the instructions to disable background battery optimizations for my phone. But even if it would be killed it should not reset the state.

siankatabg commented 2 years ago

Did you by any chance used the cli tool to upload old photos?

I've the same issue and I think the cli tool bugged my setup. I'll try to find some time and do it from scratch again to check that out.

oddlama commented 2 years ago

Yes I did use the CLI tool

dorianim commented 2 years ago

I can confirm this issue on a Huawei P30. The backup runs and as soon as the screen goes off, it stops, even though background backup is enabled.

alextran1502 commented 2 years ago

@dorianim background backup will not show the progress on the backup page as it is managed by the background worker.

dorianim commented 2 years ago

Ok, I'll leave the phone in standby for some time and check back if the pictures are uploaded.

Thanks for your great efforts, by the way! I really like this project, and it is exactly what I've been looking for for years :)

dorianim commented 2 years ago

Well... I started at 2440 assets left when I wrote the comment, left the phone in standby until now. Now, it's been at 2500 for about two minutes and then it suddenly jumped to 2370.

alextran1502 commented 2 years ago

@dorianim it means the background backup is working. It is the expected behavior, do you have any concern about this behavior?

manuke42 commented 2 years ago

If you uploaded some photos via the CLI and then try to upload some of the same photos from mobile uploads them, too, but the server rejects the upload due to same checksum and the progress will reset on next start of the app.

Steps to reproduce:

  1. Create some photos on mobile with internet turned off.
  2. upload these exact photos (assets) via another client (CLI or web)
  3. Turn on your internet on the phone
  4. The photos will upload and the progress seems finished (but rejected by server without notice in the app)
  5. Restart the app and the progress is reset and the photos will again be uploaded and agin rejected by the server.

To make this more obvious you should make many (100) photos at once and repeat the steps. So you will see the progress to reset to 100 left assets on mobile every time you restart the app.

I think the pull request #595 was going to solve this problem.

siankatabg commented 2 years ago

I'd like to report that #853 fixed the issue on my end even on the old instance where it wasn't working! Thanks.

ZuluWhiskey commented 1 year ago

I just wanted to add, for anyone else who has the same symptom (uploads not working / restarting when opening the app). I had an NGINX reverse proxy issue, so connected directly via the private IP at home and it worked. I'll edit this comment once I work out the correct configuration...

alextran1502 commented 1 year ago

I just wanted to add, for anyone else who has the same symptom (uploads not working / restarting when opening the app). I had an NGINX reverse proxy issue, so connected directly via the private IP at home and it worked. I'll edit this comment once I work out the correct configuration...

@ZuluWhiskey this is the expected behavior though. The app has two upload mechanism, foreground and background. When you out the app to the background, any current foreground upload will be canceled and if you have background upload enabled then the phone OS will schedule for the background backup to run at a later time

What is the behavior are you expecting?

ZuluWhiskey commented 1 year ago

I just wanted to add, for anyone else who has the same symptom (uploads not working / restarting when opening the app). I had an NGINX reverse proxy issue, so connected directly via the private IP at home and it worked. I'll edit this comment once I work out the correct configuration...

@ZuluWhiskey this is the expected behavior though. The app has two upload mechanism, foreground and background. When you out the app to the background, any current foreground upload will be canceled and if you have background upload enabled then the phone OS will schedule for the background backup to run at a later time

What is the behavior are you expecting?

My issue was the same as the original post, on my iPhone I would start the backup and the progress would show pictures as uploading, but if:

Then when I reopened the app it would start the upload process from the first picture again and none of the pictures that it looked like were previously uploading were anywhere to be seen on Immich. To test to see if my NGINX reverse proxy was the problem, I connected directly via the local IP and once I did so everything worked correctly 👍

So I've got an issue with my NGINX configuration, but I'm not sure what that is as I've included the headers suggested in the Immich documentation - It may just be me but I wanted to share on here just in case anyone had the same problem

ZuluWhiskey commented 1 year ago

~Ah, so looking through other issues I just spotted that you have an NGINX template in the repo which I didn't know was there 👍 I think it'd be great if that was mentioned on the Reverse Proxy page in the docs for Custom reverse proxies~

It's already there, I'm blind!

dantufis commented 1 year ago

Hello, I have this problem on Motorola edge 30. back-up don't work in background, only when the screen is on and you must be with the app open. I test it on iphone 11 and it have the same issue. All back up options are turned on. Anyway, great work with this app, I love it.

pokemonbattledroid commented 10 months ago

same issue on iphone 15 pro and pixel 7a too. maybe it's just the initial upload? just keep tapping the screen, hoping the auto upload works after the initial backup sequence.

ZuluWhiskey commented 10 months ago

same issue on iphone 15 pro and pixel 7a too. maybe it's just the initial upload? just keep tapping the screen, hoping the auto upload works after the initial backup sequence.

Are you using a direct connection with IP address only and nothing special?

pokemonbattledroid commented 10 months ago

Yes. No special setups, box standard.

-- iamgo.net

On Sat, 20 Jan 2024 at 20:49, ZuluWhiskey @.***> wrote:

same issue on iphone 15 pro and pixel 7a too. maybe it's just the initial upload? just keep tapping the screen, hoping the auto upload works after the initial backup sequence.

Are you using a direct connection with IP address only and nothing special?

— Reply to this email directly, view it on GitHub https://github.com/immich-app/immich/issues/747#issuecomment-1902261941, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK3U5ORZSSLKSDJPDUIFXXTYPQUVPAVCNFSM6AAAAAAQUXWZE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGI3DCOJUGE . You are receiving this because you commented.Message ID: @.***>