immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
49.47k stars 2.61k forks source link

Error: Invalid unit value N/A #13636

Open Brice187 opened 3 days ago

Brice187 commented 3 days ago

The bug

[Nest] 8  - 10/21/2024, 7:56:39 AM   ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): Error: Invalid unit value N/A
[Nest] 8  - 10/21/2024, 7:56:39 AM   ERROR [Microservices:JobService] Error: Invalid unit value N/A
    at asNumber (/usr/src/app/node_modules/luxon/build/node/luxon.js:2029:87)
    at normalizeObject (/usr/src/app/node_modules/luxon/build/node/luxon.js:2038:35)
    at Duration.fromObject (/usr/src/app/node_modules/luxon/build/node/luxon.js:3091:15)
    at MetadataService.getVideoTags (/usr/src/app/dist/services/metadata.service.js:537:46)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MetadataService.getExifTags (/usr/src/app/dist/services/metadata.service.js:256:67)
    at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:147:26)
    at async /usr/src/app/dist/services/job.service.js:163:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 8  - 10/21/2024, 7:56:39 AM   ERROR [Microservices:JobService] Object:
{
  "id": "42f955fe-4ab3-47cf-b752-7867932185b3"
}

The OS that Immich Server is running on

Debian Host (running in k3s v1.31.1+k3s1)

Version of Immich Server

v1.118.2

Version of Immich Mobile App

not mobile related

Platform with the issue

Your docker-compose.yml content

I am using the helm chart

Your .env content

## This chart relies on the common library chart from bjw-s
## You can find it at https://github.com/bjw-s/helm-charts/tree/main/charts/library/common
## Refer there for more detail about the supported values

# These entries are shared between all the Immich components

env:
  REDIS_HOSTNAME: '{{ printf "%s-redis-master" .Release.Name }}'
  DB_HOSTNAME: "{{ .Release.Name }}-postgresql"
  DB_USERNAME: "{{ .Values.postgresql.global.postgresql.auth.username }}"
  DB_DATABASE_NAME: "{{ .Values.postgresql.global.postgresql.auth.database }}"
  # -- You should provide your own secret outside of this helm-chart and use `postgresql.global.postgresql.auth.existingSecret` to provide credentials to the postgresql instance
  DB_PASSWORD:
    valueFrom:
      secretKeyRef:
        name: postgres-secret
        key: password
  IMMICH_MACHINE_LEARNING_URL: '{{ printf "http://%s-machine-learning:3003" .Release.Name }}'

image:
  tag: v1.118.2

immich:
  metrics:
    # Enabling this will create the service monitors needed to monitor immich with the prometheus operator
    enabled: false
  persistence:
    # Main data store for all photos shared between different components.
    library:
      # Automatically creating the library volume is not supported by this chart
      # You have to specify an existing PVC to use
      existingClaim: photos-pvc
  # configuration is immich-config.json converted to yaml
  # ref: https://immich.app/docs/install/config-file/
  #
  configuration:
    trash:
      enabled: true
      days: 3
    server:
      externalDomain: https://example.com
    notifications:
      smtp:
        enabled: true
        from: "photos@example.com"
        replyTo: "no-reply.photos@example.com"
        transport:
          ignoreCert: false
          host: "redacted.org"
          port: 587
          username: "redacted"
          password: "redacted"

    # storageTemplate:
    #   enabled: true
    #   template: "{{y}}/{{y}}-{{MM}}-{{dd}}/{{filename}}"

# Dependencies

postgresql:
  enabled: true
  image:
    repository: tensorchord/pgvecto-rs
    tag: pg14-v0.2.0
  global:
    postgresql:
      auth:
        username: immich
        database: immich
        existingSecret: postgres-secret
        secretKeys:
          userPasswordKey: password
  primary:
    #https://github.com/bitnami/charts/blob/main/bitnami/common/templates/_resources.tpl
    resourcesPreset: "small"
    containerSecurityContext:
      readOnlyRootFilesystem: false
    initdb:
      scripts:
        create-extensions.sql: |
          CREATE EXTENSION cube;
          CREATE EXTENSION earthdistance;
          CREATE EXTENSION vectors;

redis:
  enabled: true
  architecture: standalone
  auth:
    enabled: false
  master:
    persistence:
      enabled: false

# Immich components

server:
  enabled: true
  image:
    repository: ghcr.io/immich-app/immich-server
    pullPolicy: IfNotPresent
  ingress:
    main:
      enabled: true
      className: cilium
      annotations:
        cert-manager.io/cluster-issuer: "letsencrypt-prod"
      hosts:
        - host: example.com
          paths:
            - path: "/"
      tls:
        - hosts:
            - example.com
          secretName: example-tls

machine-learning:
  enabled: true
  image:
    repository: ghcr.io/immich-app/immich-machine-learning
    pullPolicy: IfNotPresent
  env:
    TRANSFORMERS_CACHE: /cache
  persistence:
    cache:
      enabled: true
      size: 10Gi
      # Optional: Set this to pvc to avoid downloading the ML models every start.
      type: emptyDir
      accessMode: ReadWriteMany
      # storageClass: your-class


### Reproduction steps

I don't know

### Relevant log output

_No response_

### Additional information

_No response_
bo0tzz commented 3 days ago

Can you share a file that this issue happens on? Ideally in a .zip

Brice187 commented 3 days ago

issue13636.zip

bo0tzz commented 3 days ago

The error is in video-related code, but that's a jpeg. Are you sure it's the correct file?

Brice187 commented 3 days ago

oops. here we go:

# check where issue happens
/tmp kubectl -n immich logs immich-server-76f48fd794-l9brc | grep -A 12 "Invalid unit value N/A" | grep '"id":'
...
  "id": "1f550aa4-c6db-49b1-87e8-5be412a95d2f"
...

# sha256 sum the correct file
root@immich-server-76f48fd794-l9brc:/usr/src/app# sha256sum upload/encoded-video/14065f40-2ef9-46a7-b562-f331a94f504b/1f/55/1f550aa4-c6db-49b1-87e8-5be412a95d2f.mp4
03e60b1795dca1a11cc14d585e211c0da29cb7ef6e15ee52cd2ebaff16ef41d9  upload/encoded-video/14065f40-2ef9-46a7-b562-f331a94f504b/1f/55/1f550aa4-c6db-49b1-87e8-5be412a95d2f.mp4

b.mp4.zip sha256: 03e60b1795dca1a11cc14d585e211c0da29cb7ef6e15ee52cd2ebaff16ef41d9

bo0tzz commented 3 days ago
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'b.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.1.100
  Duration: 00:00:00.04, start: 0.000000, bitrate: 32170 kb/s
  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(progressive), 960x720 [SAR 1:1 DAR 4:3], 32004 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 libx264

I think the issue is

Duration: 00:00:00.04

Being only a fraction of a second, but the video parsing takes the seconds component of the duration: https://github.com/immich-app/immich/blob/45517ab7aedcf488da6610587a01200decd01cbe/server/src/services/metadata.service.ts#L692-L694

Pranay-Pandey commented 3 days ago

I would like to work on this. I am new to this repo. Can you please tell me how to reproduce this? I am not able to reproduce this issue. Does this happen just for uploading this video? I am using the latest code

bo0tzz commented 3 days ago

Thanks! Yes, just uploading the video should reproduce it. Admittedly I haven't reproduced it in Immich myself, I just ran ffprobe on the file and manually looked for weird metadata (finding that subsecond Duration).

dvishal485 commented 3 days ago

I think the issue is

Duration: 00:00:00.04

Being only a fraction of a second, but the video parsing takes the seconds component of the duration:

https://github.com/immich-app/immich/blob/45517ab7aedcf488da6610587a01200decd01cbe/server/src/services/metadata.service.ts#L692-L694

But still the function fromObject even accepts fractional component of seconds.

let x = Duration.fromObject({ seconds: 0.004 }).toFormat('hh:mm:ss.SSS');
console.log(x);

yeilds 00:00:00.004.

darrelhong commented 9 hours ago

cannot reproduce this with the docker compose setup, although it looks like it is trying to process the string "N/A". thought it might be an issue with the fluent-ffmpeg library https://github.com/fluent-ffmpeg/node-fluent-ffmpeg/issues/1195. But after looking through, I believe the ffmpeg CLI is returning the string N/A instead. Wondering how are the ffmpeg binaries configured?