V1.116 - Error uploading images - `ERROR: record "new" has no field "updated_at"`

[ignaciolg]

The bug

Reopened, as is still failing on version v.1.116 using the web ui and pushing images from the phone from v.1.115

Before the 20th of September everything was working fine. Uploads with immich-go, with the phone app, etc

Now, If I try to upload new images, immich thrown an error and a broken thumbnail is being shown.

The image seems to not be written on disk (immich has write access to the upload folders, tested with podman exec -it immich_server bash and playing with the files)

The most weird thing is that the image do not exist on the disk, but I can download it from the web UI

Searching for the broken images on disk return no results

I'm not aware on any change on the configuration of any of the pieces.

Could anyone help me?

The OS that Immich Server is running on

Debian

Version of Immich Server

v1.116.0

Version of Immich Mobile App

v1.115.0

Platform with the issue

• [ ] Server
• [X] Web
• [X] Mobile

Your docker-compose.yml content

Using podman with quadlets (systemd auto-generated services)

Here are the used images for the different services

immich server - ghcr.io/immich-app/immich-server:release
immich postgres - docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
immich redis - docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
immich machine learning - ghcr.io/immich-app/immich-machine-learning:release-cuda

### Your .env content

```Shell
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION= /mnt/1.6TiB/containers/immich/upload
EXTERNAL_PATH=/mnt/1.6TiB/containers/immich/external

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=<redacted>

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=<redacted>
DB_DATABASE_NAME=<redacted>
DB_DATA_LOCATION= /mnt/1.6TiB/containers/immich/database

Reproduction steps

1. Open Immich Web
2. Upload a file
3. A modal with an error will pop up
4. Now a broken image will be shown on the timeline

Relevant log output

podman logs immich_server

Nest] 12  - 09/24/2024, 9:13:23 PM   ERROR [Api:AssetMediaService~nn50vkyr] Error uploading file QueryFailedError: record "new" has no field "updated_at"
QueryFailedError: record "new" has no field "updated_at"
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async UpdateQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/UpdateQueryBuilder.js:83:33)
    at async UserRepository.updateUsage (/usr/src/app/dist/repositories/user.repository.js:114:9)
    at async AssetMediaService.uploadAsset (/usr/src/app/dist/services/asset-media.service.js:126:13)
    at async AssetMediaController.uploadAsset (/usr/src/app/dist/controllers/asset-media.controller.js:39:29)
[Nest] 12  - 09/24/2024, 9:13:23 PM   ERROR [Api:ErrorInterceptor~nn50vkyr] Database error: QueryFailedError: record "new" has no field "updated_at"
[Nest] 2  - 09/24/2024, 9:13:23 PM   ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): Error: ENOENT: no such file or directory, stat 'upload/upload/adce3531-3d90-4b7b-b089-551338128d8a/7f/bf/7fbf58d8-5133-435f-bd5f-ec68b08e7611.webp'
[Nest] 2  - 09/24/2024, 9:13:23 PM   ERROR [Microservices:JobService] Error: ENOENT: no such file or directory, stat 'upload/upload/adce3531-3d90-4b7b-b089-551338128d8a/7f/bf/7fbf58d8-5133-435f-bd5f-ec68b08e7611.webp'
    at async Object.stat (node:internal/fs/promises:1032:18)
    at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:191:23)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 2  - 09/24/2024, 9:13:23 PM   ERROR [Microservices:JobService] Object:
{
  "id": "aec6bfca-85d7-471d-857f-f118fe94f070",
  "source": "upload"
}

podman logs immich_postgres

2024-09-24 21:12:45.284 UTC [82] STATEMENT:  INSERT INTO "audit"("entityType", "entityId", "action", "ownerId", "createdAt") VALUES ($1, $2, $3, $4, DEFAULT) RETURNING "id", "createdAt"
2024-09-24 21:13:23.740 UTC [129] ERROR:  record "new" has no field "updated_at"
2024-09-24 21:13:23.740 UTC [129] CONTEXT:  PL/pgSQL assignment "NEW.updated_at = now()"
        PL/pgSQL function change_updated_at() line 4 at assignment

### Additional information

_No response_

[alextran1502]

Can you attach a sample file that error out on upload?

[made-in-Chuguev]

The same issue with file uploads. No matter what file you upload (photo or video), you end up with this. I also noticed that if you try to delete this file, empty the recycle bin, and then try to upload it again, I get an error: 'file duplicate.' Additionally, tasks related to creating thumbnails, extracting metadata, etc., are stuck.

I also noticed that files are not being moved from 'upload'.

[ignaciolg]

Any picture is failing atm, some examples

This from web its a webp, so here is the link https://disney.images.edge.bamgrid.com/ripcut-delivery/v2/variant/disney/ba525cd2-093e-4843-a42f-a9058e80a3dd/compose?format=webp&label=hero_carousel_none_300&width=2880

This from mobile

This is failing for both users on my server.

[alextran1502]

@ignaciolg Hmm, I cannot reproduce the issue. The files can be uploaded and displayed fine. Do you mind bring the stack down, then remove the docker images and repull them?

[made-in-Chuguev]

Updated to the latest version and restarted all containers. After that, the problem went away.