immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
45.1k stars 2.19k forks source link

[BUG] Reverse geocoding repeated crash #2386

Closed truppelito closed 1 year ago

truppelito commented 1 year ago

The bug

First of all, great work. I love Immich and I'm sure this will quickly become my self-hosted photos solution.

I used the CLI bulk import from my laptop over-night and so far I imported 15000 items (photos and videos). I'm running this on a RaspberryPi 4, so I disabled video transcoding before starting the import (I then also disabled machine learning and typesense at some point and restarted the server, with the thinking of enabling them again after the import was done). I just noticed that the remaning jobs (thumbnails, metadata, migration) seemed to be hung up even though they are "Active" (example: the migration job has been stuck on Active=1, Waiting=403 for a while) so I had a look on the docker logs and found this:

[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [NestFactory] Starting Nest application...
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +226ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +3ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +26ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1476  - 05/06/2023, 12:41:09 PM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1476  - 05/06/2023, 12:41:10 PM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +455ms
[Nest] 1476  - 05/06/2023, 12:41:10 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +3ms
[Nest] 1476  - 05/06/2023, 12:41:10 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 1476  - 05/06/2023, 12:41:10 PM    WARN [MetadataExtractionProcessor] Reverse geocoding is enabled
[Nest] 1476  - 05/06/2023, 12:41:10 PM     LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding
/app/immich/server/node_modules/local-reverse-geocoder/index.js:746
            throw err;
            ^
CsvError: Invalid Record Length: expect 19, got 15 on line 7973
    at Object.__onRecord (/app/immich/server/node_modules/csv-parse/dist/cjs/index.cjs:940:11)
    at Object.parse (/app/immich/server/node_modules/csv-parse/dist/cjs/index.cjs:903:36)
    at Parser._flush (/app/immich/server/node_modules/csv-parse/dist/cjs/index.cjs:1336:26)
    at Parser.final [as _final] (node:internal/streams/transform:133:25)
    at callFinal (node:internal/streams/writable:696:27)
    at prefinish (node:internal/streams/writable:725:7)
    at finishMaybe (node:internal/streams/writable:735:5)
    at afterWrite (node:internal/streams/writable:506:3)
    at onwrite (node:internal/streams/writable:479:7)
    at Parser.Transform._read (node:internal/streams/transform:264:5) {
  code: 'CSV_RECORD_INCONSISTENT_FIELDS_LENGTH',
  bytes: 1228800,
  comment_lines: 0,
  empty_lines: 0,
  invalid_field_length: 0,
  lines: 7973,
  records: 7972,
  columns: false,
  error: undefined,
  header: false,

  index: 15,
  raw: undefined,
  column: 15,
  quoting: false,
  record: [
    '10103950',  'Windale',
    'Windale',   '',
    '-32.99277', '151.68167',
    'P',         'PPLX',
    'AU',        '',
    '02',        '14650',
    '',          '',

'
  ]
}

This appears repeated over and over again, every 20 seconds or so. I'm not sure if this reverse geocoding issue would also hang the other jobs, but I thouhgt I'd report it nevertheless.

Thank you!

The OS that Immich Server is running on

NAME="Debian GNU/Linux", VERSION="11 (bullseye)"

Version of Immich Server

v1.54.1

Version of Immich Mobile App

v1.54.0

Platform with the issue

Your docker-compose.yml content

I'm using the https://immich.app/docs/install/all-in-one compose file.

Your .env content

What the all-in-one file has, with the necessary changes for IPs/ports, databases and this:
DISABLE_MACHINE_LEANRNING=true
DISABLE_TYPESENSE=true

Reproduction steps

Explained above, but in summary:
1. Use the CLI to bulk import ~15000 items
2. Reverse geocoding has issues

Additional information

No response

truppelito commented 1 year ago

Ok, I just found this. It seems like I just posted a repeated issue. Feel free to close this if it is in fact repeated.