Pictures don't get added to albums, when uploading through cli

GitHorsting commented 6 months ago

The bug

I uploaded quite a few images through the cli, using the --recursive and --album variable. When the upload finished all images were uploaded and the albums created. Unfortunately all albums were empty and no images associated with them.

The OS that Immich Server is running on

unraid 6.12

Version of Immich Server

1.98.2

Version of Immich Mobile App

1.98.2

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

Your docker-compose.yml content

--

Your .env content

--

Reproduction steps

1. login with cli
2. immich upload --album --recursive directory/
3. output: Error 500

Additional information

All assets were already uploaded, nothing to do. Creating albums | ████████████████████████████████████████ | 100% | ETA: 0s | 0/0 albums Adding assets to albums | █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 3% | ETA: 26s | 1833/54927 assets file:///usr/local/lib/node_modules/@immich/cli/dist/index.js:9798 throw new y(e.status, e.data, e.headers); ^

y [Error]: Error: 500 at Object.l [as ok] (file:///usr/local/lib/node_modules/@immich/cli/dist/index.js:9798:9) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async UploadCommand.updateAlbums (file:///usr/local/lib/node_modules/@immich/cli/dist/index.js:18394:11) at async UploadCommand.run (file:///usr/local/lib/node_modules/@immich/cli/dist/index.js:18230:56) at async Command. (file:///usr/local/lib/node_modules/@immich/cli/dist/index.js:18501:3) { status: 500, data: { message: 'Failed to add assets to album', error: 'Internal Server Error', statusCode: 500 }, headers: Headers { [Symbol(headers list)]: HeadersList { cookies: null, [Symbol(headers map)]: Map(7) { 'x-powered-by' => { name: 'X-Powered-By', value: 'Express' }, 'content-type' => { name: 'Content-Type', value: 'application/json; charset=utf-8' }, 'content-length' => { name: 'Content-Length', value: '92' }, 'etag' => { name: 'ETag', value: '"5c-lyaVQAHBg1LfX0s5U5+HqLAWa+A"' }, 'date' => { name: 'Date', value: 'Mon, 18 Mar 2024 18:47:12 GMT' }, 'connection' => { name: 'Connection', value: 'keep-alive' }, 'keep-alive' => { name: 'Keep-Alive', value: 'timeout=5' } },

},
[Symbol(guard)]: 'immutable',
[Symbol(realm)]: null

} }

Node.js v20.10.0

GitHorsting commented 6 months ago

BTW, I'm using CLI version 2.1.0

schuhbacca commented 6 months ago

Can you try updating? Cli is on version 2.8.0 at this point I think

dkuwahara commented 6 months ago

I'm seeing the same thing on my end. The following logs appear on the server side.

[Nest] 7 - 03/18/2024, 10:50:02 PM ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180" at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) at async AlbumRepository.addAssets (/usr/src/app/dist/infra/repositories/album.repository.js:204:9) at async AlbumService.addAssets (/usr/src/app/dist/domain/album/album.service.js:164:13)] Failed to add assets to album [Nest] 7 - 03/18/2024, 10:50:02 PM ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180" at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) at async AlbumRepository.addAssets (/usr/src/app/dist/infra/repositories/album.repository.js:204:9) at async AlbumService.addAssets (/usr/src/app/dist/domain/album/album.service.js:164:13)] QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"

schuhbacca commented 6 months ago

Sorry was incorrect about the versioning. Issue was raised yesterday though. Suggested workaround in meantime is use immich-go.

https://github.com/immich-app/immich/issues/8028

chengguangnan commented 6 months ago

I have the same error on cli 2.2.0

marciopamplona commented 6 months ago

Same problem on CLI 2.2.0

ulilicht commented 6 months ago

Not sure if its the same issue, but I as well get a 500 error when uploading a. bulk of albums using the CLI Tool:

ulilicht@FVFX31XXHV2H Reisen % immich upload --recursive --album ./                                                       
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 6758/6758 assets
Found 6676 new files and 82 duplicates
Uploading assets | ████████████████████████████████████████ | 100% | ETA: 0s | 42,7 GB/42,7 GB
Successfully uploaded 6676 assets (42,7 GB)
Creating albums | ████████████████████████████████████████ | 100% | ETA: 0s | 47/47 albums
Successfully created 47 new albums
Successfully updated 6758 assets
Adding assets to albums | ████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 9% | ETA: 19s | 649/6758 assets
file:///Users/ulilicht/.nvm/versions/node/v20.12.0/lib/node_modules/@immich/cli/dist/index.js:3064
  throw new y(e.status, e.data, e.headers);
        ^

y [Error]: Error: 500
    at Object.l [as ok] (file:///Users/ulilicht/.nvm/versions/node/v20.12.0/lib/node_modules/@immich/cli/dist/index.js:3064:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async updateAlbums (file:///Users/ulilicht/.nvm/versions/node/v20.12.0/lib/node_modules/@immich/cli/dist/index.js:18020:9)
    at async upload (file:///Users/ulilicht/.nvm/versions/node/v20.12.0/lib/node_modules/@immich/cli/dist/index.js:17812:3) {
  status: 500,
  data: {
    message: 'Failed to add assets to album',
    error: 'Internal Server Error',
    statusCode: 500
  },
  headers: Headers {
    [Symbol(headers list)]: HeadersList {
      cookies: null,
      [Symbol(headers map)]: Map(7) {
        'x-powered-by' => { name: 'X-Powered-By', value: 'Express' },
        'content-type' => {
          name: 'Content-Type',
          value: 'application/json; charset=utf-8'
        },
        'content-length' => { name: 'Content-Length', value: '92' },
        'etag' => { name: 'ETag', value: '"5c-lyaVQAHBg1LfX0s5U5+HqLAWa+A"' },
        'date' => { name: 'Date', value: 'Tue, 02 Apr 2024 07:32:44 GMT' },
        'connection' => { name: 'Connection', value: 'keep-alive' },
        'keep-alive' => { name: 'Keep-Alive', value: 'timeout=5' }
      },
      [Symbol(headers map sorted)]: null
    },
    [Symbol(guard)]: 'immutable',
    [Symbol(realm)]: null
  }
}

Node.js v20.12.0

rollacting commented 6 months ago

Hi everyone. I experienced the same issue and - in my case - was able to narrow it down to two .MOV files having different filenames, but being the same video/content.

It is pretty easy to reproduce:

create a new folder 2024-04-05_TEST and add the two images attached below
- Cc.logo.circle.png
- Cc.logo.circle copy.png
via immich-cli, run immich upload --album /my-import-path/2024-04-05_TEST
result:
- one of the two images gets added to the foto library and is visible in the webui after import
- an empty album with the name "2024-04-05_TEST" is created (it contains no images)
- the raised postgres unique constraint violation (see https://github.com/immich-app/immich/issues/8046#issuecomment-2005197256 ) is printed in the server log
- the cli command returns the 500 exception message from the original post https://github.com/immich-app/immich/issues/8046#issue-2193021449

Jetro97 commented 5 months ago

Same error here, with large albums (> 100 photos).

volkovd commented 5 months ago

I've got same issue. Stacktrace from immich-server:

[Nest] 7  - 04/12/2024, 4:17:40 PM   ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async AlbumRepository.addAssetIds (/usr/src/app/dist/repositories/album.repository.js:205:9)
    at async addAssets (/usr/src/app/dist/utils/asset.util.js:29:9)
    at async AlbumService.addAssets (/usr/src/app/dist/services/album.service.js:148:25)] Failed to add assets to album
[Nest] 7  - 04/12/2024, 4:17:40 PM   ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async AlbumRepository.addAssetIds (/usr/src/app/dist/repositories/album.repository.js:205:9)
    at async addAssets (/usr/src/app/dist/utils/asset.util.js:29:9)
    at async AlbumService.addAssets (/usr/src/app/dist/services/album.service.js:148:25)] QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"

P4R4DiSi4C commented 5 months ago

Same issue here

arif-basri commented 5 months ago

Same here.

Version of Immich Server 1.103.1

Immich CLI 2.2.0


[Nest] 8  - 05/02/2024, 5:40:38 AM   ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async AlbumRepository.addAssetIds (/usr/src/app/dist/repositories/album.repository.js:216:9)
    at async addAssets (/usr/src/app/dist/utils/asset.util.js:29:9)
    at async AlbumService.addAssets (/usr/src/app/dist/services/album.service.js:154:25)] Failed to add assets to album
[Nest] 8  - 05/02/2024, 5:40:38 AM   ERROR [QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async AlbumRepository.addAssetIds (/usr/src/app/dist/repositories/album.repository.js:216:9)
    at async addAssets (/usr/src/app/dist/utils/asset.util.js:29:9)
    at async AlbumService.addAssets (/usr/src/app/dist/services/album.service.js:154:25)] QueryFailedError: duplicate key value violates unique constraint "PK_c67bc36fa845fb7b18e0e398180"

arif@minta:~$ immich upload --album-name "iphone12mini" --recursive ./
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 61s
Found 6100 new files and 0 duplicates
Uploading assets | ████████████████████████████████████████ | 100% | ETA: 0s | B
Successfully uploaded 6100 assets (85.5 GB)
Creating albums | ████████████████████████████████████████ | 100% | ETA: 0s | 1s
Successfully created 1 new album
Successfully updated 6100 assets
Adding assets to albums | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: s
file:///home/arif/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:3064
  throw new y(e.status, e.data, e.headers);
        ^

y [Error]: Error: 500
    at Object.l [as ok] (file:///home/arif/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:3064:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async updateAlbums (file:///home/arif/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:18020:9)
    at async upload (file:///home/arif/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:17812:3) {
  status: 500,
  data: {
    message: 'Failed to add assets to album',
    error: 'Internal Server Error',
    statusCode: 500
  },
  headers: Headers {
    [Symbol(headers list)]: HeadersList {
      cookies: null,
      [Symbol(headers map)]: Map(8) {
        'x-powered-by' => { name: 'x-powered-by', value: 'Express' },
        'x-immich-cid' => { name: 'x-immich-cid', value: 'z4jcqf49' },
        'content-type' => {
          name: 'content-type',
          value: 'application/json; charset=utf-8'
        },
        'content-length' => { name: 'content-length', value: '92' },
        'etag' => { name: 'etag', value: '"5c-lyaVQAHBg1LfX0s5U5+HqLAWa+A"' },
        'date' => { name: 'date', value: 'Thu, 02 May 2024 05:40:38 GMT' },
        'connection' => { name: 'connection', value: 'keep-alive' },
        'keep-alive' => { name: 'keep-alive', value: 'timeout=5' }
      },
      [Symbol(headers map sorted)]: null
    },
    [Symbol(guard)]: 'immutable',
    [Symbol(realm)]: null
  }
}

Node.js v21.7.1

gullygossner commented 4 months ago

in a similar boat

Version of Immich Server 1.103.1

Immich CLI 2.2.0

Successfully created 8 new albums Successfully updated 13560 assets Adding assets to albums | █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 1% | ETA: 22s | 209/13560 assets file:///app/immich/cli/dist/index.js:3029 throw new y(e.status, e.data, e.headers); ^

y [Error]: Error: 500 at Object.l [as ok] (file:///app/immich/cli/dist/index.js:3029:9) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async updateAlbums (file:///app/immich/cli/dist/index.js:16522:9) at async upload (file:///app/immich/cli/dist/index.js:16298:3) { status: 500, data: { message: 'Failed to add assets to album', error: 'Internal Server Error', statusCode: 500 }, headers: Headers {

[Symbol(headers list)]: HeadersList {
  cookies: null,
  [Symbol(headers map)]: Map(8) {
    'x-powered-by' => { name: 'X-Powered-By', value: 'Express' },
    'x-immich-cid' => { name: 'x-immich-cid', value: 'q5zx8tj8' },
    'content-type' => {
      name: 'Content-Type',
      value: 'application/json; charset=utf-8'
    },
    'content-length' => { name: 'Content-Length', value: '92' },
    'etag' => { name: 'ETag', value: '"5c-lyaVQAHBg1LfX0s5U5+HqLAWa+A"' },
    'date' => { name: 'Date', value: 'Mon, 13 May 2024 12:26:05 GMT' },
    'connection' => { name: 'Connection', value: 'keep-alive' },
    'keep-alive' => { name: 'Keep-Alive', value: 'timeout=5' }
  },
  [Symbol(headers map sorted)]: null
},
[Symbol(realm)]: null

} }

Node.js v20.12.2

marciopamplona commented 3 months ago

@etnoy can you take a look at this? It seems the cli is a bit forgotten... Some of us depends on mass operation by using the cli interface. Thank you.

etnoy commented 3 months ago

I've been a bit away from immich due to a construction project. Sorry, I know that the cli sometimes falls off the wayside, especially now that I'm temporarily busy with other things. Feel free to continue debugging or send a PR, the cli is not hard to work on