search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
51.96k stars 2.75k forks source link

[BUG] Unable to use read-only gallery import since 1.79 #4307

Closed andrejohansson closed 1 year ago

andrejohansson commented 1 year ago

The bug

My server emits smtp authentication errors and floods my logs in /var/log/mail.log

sSMTP[2423065]: at createError (/usr/src/app/node_modules/immich/node_modules/axios/lib/core/createError.js:16:15)"

Having outgoing emails with errors and potential insecure stack traces is a risk that I would like to opt into, not happen without my knowledge. Have I missed some setting perhaps?

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.80.0

Version of Immich Mobile App

v1.79.0 build.103

Platform with the issue

Your docker-compose.yml content

Unable to share public. But using original compose + mounted read only volume + traefik proxy in front.

Your .env content

###################################################################################
# Database
###################################################################################

# NOTE: The following four database variables support Docker secrets by adding a *_FILE suffix to the variable name
# See the docker-compose documentation on secrets for additional details: https://docs.docker.com/compose/compose-file/compose-file-v3/#secrets
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# REDIS_URL will be used to pass custom options to ioredis.
# Example for Sentinel
# {"sentinels":[{"host":"redis-sentinel-node-0","port":26379},{"host":"redis-sentinel-node-1","port":26379},{"host":"redis-sentinel-node-2","port":26379}],"name":"redis-sentinel"}
# REDIS_URL=ioredis://eyJzZW50aW5lbHMiOlt7Imhvc3QiOiJyZWRpcy1zZW50aW5lbDEiLCJwb3J0IjoyNjM3OX0seyJob3N0IjoicmVkaXMtc2VudGluZWwyIiwicG9ydCI6MjYzNzl9XSwibmFtZSI6Im15bWFzdGVyIn0=

# Optional Redis settings:

# Note: these parameters are not automatically passed to the Redis Container
# to do so, please edit the docker-compose.yml file as well. Redis is not configured
# via environment variables, only redis.conf or the command line

# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_USERNAME=
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/mnt/datapool/immich-upload

###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=<removed>
# TYPESENSE_ENABLED=false
# TYPESENSE_URL uses base64 encoding for the nodes json.
# Example JSON that was used:
# [
#      { "host": "typesense-1.example.net", "port": "443", "protocol": "https" },
#      { "host": "typesense-2.example.net", "port": "443", "protocol": "https" },
#      { "host": "typesense-3.example.net", "port": "443", "protocol": "https" },
# ]
# TYPESENSE_URL=ha://WwogIHsgImhvc3QiOiAidHlwZXNlbnNlLTEuZXhhbXBsZS5uZXQiLCAicG9ydCI6ICI0NDMiLCAicHJvdG9jb2wiOiAiaHR0cHMiIH0sCiAgeyAiaG9zdCI6ICJ0eXBlc2Vuc2UtMi5leGFtcGxlLm5ldCIsICJwb3J0IjogIjQ0MyIsICJwcm90b2NvbCI6ICJodHRwcyIgfSwKICB7ICJob3N0IjogInR5cGVzZW5zZS0zLmV4YW1wbGUubmV0IiwgInBvcnQiOiAiNDQzIiwgInByb3RvY29sIjogImh0dHBzIiB9Cl0=

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

###################################################################################
# Immich Version - Optional
#
# This allows all immich docker images to be pinned to a specific version. By default,
# the version is "release" but could be a specific version, like "v1.59.0".
###################################################################################

IMMICH_VERSION=v1.80.0

Reproduction steps

Doing nothing especially, logs are flooded many messages per second. 

Worth nothing is that I run a periodic cron job to populate my read-only gallery setup (every 2h) using the instructions from the [documentation](https://documentation.immich.app/docs/features/read-only-gallery).

Additional information

image

bo0tzz commented 1 year ago

Immich doesn't have any SMTP related code and doesn't do any sort of email sending in general. The error logs you post are a bit strange and don't really have much information. Are there any other logs? Are you seeing any unusual behaviour when using Immich?

andrejohansson commented 1 year ago

I digged some more, it comes from my scheduled job that tries to import new assets (which have been running since the 1.79 update). I updated the title of this since the smtp part seems to be my servers default config. But it seems that some library id must be provided also now?

Script command

docker exec -i immich_server /bin/sh -c "immich upload --key <my-key> --server http://immich_proxy:8080/api /mnt/media/familjen --recursive --import -y"

Response error:

Oct  2 16:31:11 singularity sSMTP[2258465]:   response: {
Oct  2 16:31:11 singularity sSMTP[2258465]:     message: [
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId must be a UUID',
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId should not be empty',
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId must be a string'
Oct  2 16:31:11 singularity sSMTP[2258465]:     ],
Oct  2 16:31:11 singularity sSMTP[2258465]:     error: 'Bad Request',
Oct  2 16:31:11 singularity sSMTP[2258465]:     statusCode: 400
Oct  2 16:31:11 singularity sSMTP[2258465]:   }

A full log entry

Oct  2 16:31:11 singularity sSMTP[2258465]: Error asset:  {
Oct  2 16:31:11 singularity sSMTP[2258465]:   file: '/mnt/media/familjen/2018/Aug/IMG_4336.MOV',
Oct  2 16:31:11 singularity sSMTP[2258465]:   reason: Error: Request failed with status code 400
Oct  2 16:31:11 singularity sSMTP[2258465]:       at createError (/usr/src/app/node_modules/immich/node_modules/axios/lib/core/createError.js:16:15)
Oct  2 16:31:11 singularity sSMTP[2258465]:       at settle (/usr/src/app/node_modules/immich/node_modules/axios/lib/core/settle.js:17:12)
Oct  2 16:31:11 singularity sSMTP[2258465]:       at IncomingMessage.handleStreamEnd (/usr/src/app/node_modules/immich/node_modules/axios/lib/adapters/http.js:322:11)
Oct  2 16:31:11 singularity sSMTP[2258465]:       at IncomingMessage.emit (node:events:526:35)
Oct  2 16:31:11 singularity sSMTP[2258465]:       at endReadableNT (node:internal/streams/readable:1359:12)
Oct  2 16:31:11 singularity sSMTP[2258465]:       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
Oct  2 16:31:11 singularity sSMTP[2258465]:     config: {
Oct  2 16:31:11 singularity sSMTP[2258465]:       transitional: [Object],
Oct  2 16:31:11 singularity sSMTP[2258465]:       adapter: [Function: httpAdapter],
Oct  2 16:31:11 singularity sSMTP[2258465]:       transformRequest: [Array],
Oct  2 16:31:11 singularity sSMTP[2258465]:       transformResponse: [Array],
Oct  2 16:31:11 singularity sSMTP[2258465]:       timeout: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       xsrfCookieName: 'XSRF-TOKEN',
Oct  2 16:31:11 singularity sSMTP[2258465]:       xsrfHeaderName: 'X-XSRF-TOKEN',
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxContentLength: Infinity,
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxBodyLength: Infinity,
Oct  2 16:31:11 singularity sSMTP[2258465]:       validateStatus: [Function: validateStatus],
Oct  2 16:31:11 singularity sSMTP[2258465]:       headers: [Object],
Oct  2 16:31:11 singularity sSMTP[2258465]:       method: 'post',
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxRedirects: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       url: 'http://immich_proxy:8080/api/asset/import',
Oct  2 16:31:11 singularity sSMTP[2258465]:       data: '{"deviceAssetId":"IMG_4336.MOV-3400878","deviceId":"CLI","fileCreatedAt":"2018-04-22T09:34:32.000Z","fileModifiedAt":"2018-04-22T09:34:32.000Z","isFavorite":"false","fileExtension":".MOV","duration":"0:00:00.000000","isReadOnly":"true","assetPath":"/mnt/media/familjen/2018/Aug/IMG_4336.MOV"}'
Oct  2 16:31:11 singularity sSMTP[2258465]:     },
Oct  2 16:31:11 singularity sSMTP[2258465]:     request: ClientRequest {
Oct  2 16:31:11 singularity sSMTP[2258465]:       _events: [Object: null prototype],
Oct  2 16:31:11 singularity sSMTP[2258465]:       _eventsCount: 3,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _maxListeners: undefined,
Oct  2 16:31:11 singularity sSMTP[2258465]:       outputData: [],
Oct  2 16:31:11 singularity sSMTP[2258465]:       outputSize: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       writable: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       destroyed: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _last: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       chunkedEncoding: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       shouldKeepAlive: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxRequestsOnConnectionReached: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _defaultKeepAlive: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       useChunkedEncodingByDefault: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       sendDate: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _removedConnection: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _removedContLen: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _removedTE: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       strictContentLength: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _contentLength: 292,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _hasBody: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _trailer: '',
Oct  2 16:31:11 singularity sSMTP[2258465]:       finished: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _headerSent: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _closed: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       socket: [Socket],
Oct  2 16:31:11 singularity sSMTP[2258465]:       _header: 'POST /api/asset/import HTTP/1.1\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'Accept: application/json, text/plain, */*\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'Content-Type: application/json\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'x-api-key: RETRACTED' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'User-Agent: axios/0.26.1\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'Content-Length: 292\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'Host: immich_proxy:8080\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         'Connection: close\r\n' +
Oct  2 16:31:11 singularity sSMTP[2258465]:         '\r\n',
Oct  2 16:31:11 singularity sSMTP[2258465]:       _keepAliveTimeout: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       _onPendingData: [Function: nop],
Oct  2 16:31:11 singularity sSMTP[2258465]:       agent: [Agent],
Oct  2 16:31:11 singularity sSMTP[2258465]:       socketPath: undefined,
Oct  2 16:31:11 singularity sSMTP[2258465]:       method: 'POST',
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxHeaderSize: undefined,
Oct  2 16:31:11 singularity sSMTP[2258465]:       insecureHTTPParser: undefined,
Oct  2 16:31:11 singularity sSMTP[2258465]:       joinDuplicateHeaders: undefined,
Oct  2 16:31:11 singularity sSMTP[2258465]:       path: '/api/asset/import',
Oct  2 16:31:11 singularity sSMTP[2258465]:       _ended: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:       res: [IncomingMessage],
Oct  2 16:31:11 singularity sSMTP[2258465]:       aborted: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       timeoutCb: null,
Oct  2 16:31:11 singularity sSMTP[2258465]:       upgradeOrConnect: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       parser: null,
Oct  2 16:31:11 singularity sSMTP[2258465]:       maxHeadersCount: null,
Oct  2 16:31:11 singularity sSMTP[2258465]:       reusedSocket: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       host: 'immich_proxy',
Oct  2 16:31:11 singularity sSMTP[2258465]:       protocol: 'http:',
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kCapture)]: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kBytesWritten)]: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kNeedDrain)]: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(corked)]: 0,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kOutHeaders)]: [Object: null prototype],
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(errored)]: null,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kHighWaterMark)]: 16384,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kRejectNonStandardBodyWrites)]: false,
Oct  2 16:31:11 singularity sSMTP[2258465]:       [Symbol(kUniqueHeaders)]: null
Oct  2 16:31:11 singularity sSMTP[2258465]:     },
Oct  2 16:31:11 singularity sSMTP[2258465]:     response: {
Oct  2 16:31:11 singularity sSMTP[2258465]:       status: 400,
Oct  2 16:31:11 singularity sSMTP[2258465]:       statusText: 'Bad Request',
Oct  2 16:31:11 singularity sSMTP[2258465]:       headers: [Object],
Oct  2 16:31:11 singularity sSMTP[2258465]:       config: [Object],
Oct  2 16:31:11 singularity sSMTP[2258465]:       request: [ClientRequest],
Oct  2 16:31:11 singularity sSMTP[2258465]:       data: [Object]
Oct  2 16:31:11 singularity sSMTP[2258465]:     },
Oct  2 16:31:11 singularity sSMTP[2258465]:     isAxiosError: true,
Oct  2 16:31:11 singularity sSMTP[2258465]:     toJSON: [Function: toJSON]
Oct  2 16:31:11 singularity sSMTP[2258465]:   },
Oct  2 16:31:11 singularity sSMTP[2258465]:   response: {
Oct  2 16:31:11 singularity sSMTP[2258465]:     message: [
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId must be a UUID',
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId should not be empty',
Oct  2 16:31:11 singularity sSMTP[2258465]:       'libraryId must be a string'
Oct  2 16:31:11 singularity sSMTP[2258465]:     ],
Oct  2 16:31:11 singularity sSMTP[2258465]:     error: 'Bad Request',
Oct  2 16:31:11 singularity sSMTP[2258465]:     statusCode: 400
Oct  2 16:31:11 singularity sSMTP[2258465]:   }
Oct  2 16:31:11 singularity sSMTP[2258465]: }
alextran1502 commented 1 year ago

We will be deprecating read-only feature to support the newly added library feature. Please consider migrating to the new library approach

jrasm91 commented 1 year ago

This is also fixed in 1.80. Let us know if you still run into the issue after upgrading to the latest release!