immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
52.76k stars 2.8k forks source link

Only on wifi background backup sometimes uses mobile data #12628

Open vyPal opened 2 months ago

vyPal commented 2 months ago

The bug

Hello, I was notified today by my phone, that some app was using a lot of mobile data in the background. When I checked, it said that immich had used around 1,5gb of mobile data in the background (and another 50mb in the foreground, but that was intentional). I checked to make sure, but I have the background backup set to only on wifi. While this luckily isn't much of an issue for me, I feel like this could be more problematic for users that don't have large data plans.

The OS that Immich Server is running on

Ubuntu 22.04.4 LTS

Version of Immich Server

v1.114.0

Version of Immich Mobile App

V1.114.0

Platform with the issue

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${THUMB_LOCATION}:/usr/src/app/upload/thumbs
      - ${ENCODED_VIDEO_LOCATION}:/usr/src/app/upload/encoded-video
      - ${PROFILE_LOCATION}:/usr/src/app/upload/profile
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

UPLOAD_LOCATION=/mnt/Lacie4TB/immich/library
THUMB_LOCATION=/mnt/Lacie4TB/immich/thumbs
ENCODED_VIDEO_LOCATION=/mnt/Lacie4TB/immich/encoded-video
PROFILE_LOCATION=/mnt/Lacie4TB/immich/profile
DB_DATA_LOCATION=/mnt/Lacie4TB/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

I guess just have mobile data on?

Relevant log output

No response

Additional information

If I have some time alter today, I might go through the client code and try to find the issue myself, I have some flutter experience, so I might be able to fix it

vyPal commented 2 months ago

I believe I have an explanation for this, when I have both mobile data and wifi turned on, if my phone connects to a wifi network that requires some additional login or conformation via a captive portal (or something similar), the apps see the status as being connected to wifi, which is true, but since some login is required, the phone still actually uses mobile data.

Samaze123 commented 2 months ago

I have the same issue here

gschintgen commented 2 months ago

Duplicate of https://github.com/immich-app/immich/issues/2075?

vyPal commented 2 months ago

I think it has the same cause, but it's not the exact same issue, in what you are referencing it continues in the backup even after leaving the wifi, in my case it starts the backup when I connect to a wifi, even when the wifi doesn't actually have internet access, and the phone is actually using data

vyPal commented 2 months ago

I still have to test this, but I think the problem is the outdated version of the connectivity_plus library

g0d-m0nky commented 1 month ago

This just happened to me and needless to say, I'm pretty angry about it. I realised when I got a message from my carrier saying I had used an extra $30 of data in a morning.

This is very dangerous, and while I don't want to slam the dev team too hard as overall it's an amazing FOSS project, this is a serious issue that could cost someone a lot of money. The app must be pulled until this is rectified otherwise someone is going to seriously burnt.

vyPal commented 1 month ago

I'm still trying to see what exactly this is caused by, my suspicions are either the outdated version of the connectivity_plus library that is being used, or something to do with how androidx.work (what handles background workers) classifies networks.

Anyhow, for now I'd recommend users to disable mobile data access for immich through their phone's settings app.

I hope someone can find a fix soon, I am not one of the devs of this project, but I have spent some time looking around trying to find the exact cause. No luck yet

alextran1502 commented 1 month ago

@g0d-m0nky hey man, sorry about that. Which phone are you using Android or iOS?

g0d-m0nky commented 1 month ago

@g0d-m0nky hey man, sorry about that. Which phone are you using Android or iOS?

Hi, I'm on Android, Samsung S23U.

interunivers commented 1 month ago

Noticing the same behaviour sometimes on iOS as well. I am on iOS 18.1.

Samaze123 commented 1 month ago

I am also on iOS 18.0 and get the same behavior

vyPal commented 1 month ago

Ok so since you said that you are having issues with this on iOS as well, I have looked through the native code for the background service workers for both iOS and Android, both seem to handle checking for WiFi connectivity about like this: (iOS example, which is okay if we're only checking for a connection to an unmetered wifi, which is usually the case, but it the case of this issue, it would also be appropriate to either check that wifi is the only connection available, or filter to make sure that mobile data is not an available connection type.

Samaze123 commented 1 month ago

The other day, while I was out for a walk, I saw my phone start to save photos. Here a screenshot (don't know if it could help):

IMG_0557
ashishjullia commented 2 weeks ago

Same issue, fyi here are the details I provided on official discord https://discord.com/channels/979116623879368755/1303432146513297481

But I'll paste the same details here with the same wording.

I'm a little confused here maybe the answer is "iOS's implementation" but I'll still write it here to get more insights.

I want to let immich only upload files while being on "Wi-Fi only" but even if I select the "Only on WiFi" under "Automatic background backup is on" in "Backup Options" -> it still uploads files when I'm not on Wi-Fi and my current understanding it that it is because of iOS level "Wi-Fi & Cellular Data" and I don't want to switch the iOS setting to "Wi-Fi" only because in that case the background app refresh for new data won't work for other applications as well.

Seems like a chicken-egg problem to me.

Am I missing something? Any possible solution for this? Are there any active issues for this?

image

The above photo is just for the reference for iOS settings but not my actual settings, they are shown below.

I don’t think it is working as expected

image image image

It is uploading on “cellular data” and I cannot set “Wi-Fi” only in iOS global background app refresh as I need the apps to refresh on both.

johachi commented 1 day ago

I have the same problem as well and have attached a video as it is happening every time I take a photo or record a video.

App Version: 1.121.0 build.184 Server Version: 1.121.0 Phone Model: iPhone 13 iOS Version: 18.1

I also have Tailscale running on my phone and NAS. The IP to the Immich server that I have entered in the phone app is the Tailscale IP provided IP.

https://github.com/user-attachments/assets/19f9b4bd-e4a5-4296-9d7c-a649e869681b

gschintgen commented 1 day ago

@johachi :

attached a video

Isn't that video just showing foreground backup working as intended?

If foreground backup is enabled: whenever the app is opened or resumed, it will check if any photos or videos in the selected album(s) have yet to be uploaded to the cloud (the remainder count). If there are any, they will be uploaded.

johachi commented 1 day ago

@johachi :

attached a video

Isn't that video just showing foreground backup working as intended?

If foreground backup is enabled: whenever the app is opened or resumed, it will check if any photos or videos in the selected album(s) have yet to be uploaded to the cloud (the remainder count). If there are any, they will be uploaded.

Hmmm, I wonder if you might be correct. I expected the WiFi option to be respected by both foreground and background but I see now that it is actually specifically under the background option.

I have now turned the foreground option off and it seems to have stopped. Thanks! I hope this was it for me!