immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
42.56k stars 2.08k forks source link

[BUG] Geolocation is wrong #2669

Closed jkasten80 closed 1 year ago

jkasten80 commented 1 year ago

The bug

The geolocation of the pictures is wrong. The map is correct but the title says a location 5-10km away from the location in the map.

The OS that Immich Server is running on

Synology NAS

Version of Immich Server

v1.60

Version of Immich Mobile App

v1.60.0 build 100

Platform with the issue

Your docker-compose.yml content

version: "3.9"
services:
  immich-redis:
    image: redis
    container_name: Immich-REDIS
    hostname: immich-redis
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping || exit 1"]
    user: 1027:100
    environment:
      - TZ=Europe/Berlin
    volumes:
      - /volume2/docker/immich/redis:/data
    restart: on-failure:5

  immich-db:
    image: postgres
    container_name: Immich-DB
    hostname: immich-db
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: ["CMD", "pg_isready", "-q", "-d", "immich", "-U", "immichuser"]
      interval: 10s
      timeout: 5s
      retries: 5
    user: 1027:100
    volumes:
      - /volume2/docker/immich/db:/var/lib/postgresql/data
    environment:
      - TZ=Europe/Berlin
      - POSTGRES_DB=immich
      - POSTGRES_USER=immichuser
      - POSTGRES_PASSWORD=immichpw
    restart: on-failure:5

  immich-server:
    image: altran1502/immich-server:release
    command: ["start-server.sh"]
    container_name: Immich-SERVER
    hostname: immich-server
    user: 1027:100
    security_opt:
      - no-new-privileges:true
    env_file:
      - stack.env
    volumes:
      - /volume2/docker/immich/upload:/usr/src/app/upload
    restart: on-failure:5
    depends_on:
      immich-redis:
        condition: service_healthy
      immich-db:
        condition: service_started
      typesense:
        condition: service_started

  immich-microservices:
    image: altran1502/immich-server:release

    command: ["start-microservices.sh"]
    container_name: Immich-MICROSERVICES
    hostname: immich-microservices
    user: 1027:100
    security_opt:
      - no-new-privileges:true
    env_file:
      - stack.env
    volumes:
      - /volume2/docker/immich/upload:/usr/src/app/upload
      - /volume2/docker/immich/micro:/usr/src/app/.reverse-geocoding-dump
    restart: on-failure:5
    depends_on:
      immich-redis:
        condition: service_healthy
      immich-db:
        condition: service_started
      typesense:
        condition: service_started

  immich-machine-learning:
    image: altran1502/immich-machine-learning:release
    container_name: Immich-LEARNING
    hostname: immich-machine-learning
    user: 1027:100
    security_opt:
      - no-new-privileges:true
    env_file:
      - stack.env
    volumes:
      - /volume2/docker/immich/upload:/usr/src/app/upload
      - /volume2/docker/immich/cache:/cache
    restart: on-failure:5
    depends_on:
      immich-db:
        condition: service_started

  immich-web:
    image: altran1502/immich-web:release
    container_name: Immich-WEB
    hostname: immich-web
    user: 1027:100
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:3000/ || exit 1
    env_file:
      - stack.env
    restart: on-failure:5
    depends_on:
      - immich-server

  immich-proxy:
    image: altran1502/immich-proxy:release
    logging:
      driver: none
    container_name: Immich-PROXY
    hostname: immich-proxy
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: curl -f http://localhost:8080/ || exit 1
    environment:
      - IMMICH_SERVER_URL=http://immich-server:3001
      - IMMICH_WEB_URL=http://immich-web:3000
    ports:
      - 8212:8080
    restart: on-failure:5
    depends_on:
      - immich-server

  typesense:
    container_name: Immich-TYPESENSE
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=meinapikeyxxx
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - /volume2/docker/immich/typesense:/data
    restart: on-failure:5

Your .env content

NODE_ENV=production
TZ=Europe/Berlin
TYPESENSE_API_KEY=meinapikeyxxx
###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich-db
DB_USERNAME=immichuser
DB_PASSWORD=immichpw
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich-redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Log message level - [simple|verbose]
###################################################################################

LOG_LEVEL=simple

###################################################################################
# JWT SECRET
###################################################################################

# This JWT_SECRET is used to sign the authentication keys for user login
# You should set it to a long randomly generated value
# You can use this command to generate one: openssl rand -base64 128
JWT_SECRET=xia7jNMRT50PgcShC0BTuFxTLPamoYnQO8OKH8jUff4iD0AdtbC8HTZp/Rv4LSdpV7kXbYTvxNrw24NzJJiag2nisYbUmEV8XbJEEboag98z+6KR5djhPPC4/oztoSJHOB2rpPXjv4VNwCAT0PvTKUo3gSY8rDja85DGugYqJ0c=

###################################################################################
# Reverse Geocoding
####################################################################################

# DISABLE_REVERSE_GEOCODING=false

# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM

DISABLE_REVERSE_GEOCODING=false
REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
####################################################################################

# Custom message on the login page, should be written in HTML form.
# For example PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# OAuth Setting - Optional
#
# These setting will enable OAuth login for your instance of Immich
# Folow the instructions in the page https://immich.app/docs/usage/oauth to set up your OAuth provider
####################################################################################

# OAUTH_ENABLED=false
# OAUTH_ISSUER_URL=
# OAUTH_CLIENT_ID=
# OAUTH_CLIENT_SECRET=
# OAUTH_BUTTON_TEXT=Login with OAuth
# OAUTH_AUTO_REGISTER=true
# OAUTH_SCOPE="openid profile email"

Reproduction steps

I just checked the geolocation on the map and the name direct above on different pics. The map is always correct, the name of the location not.

Additional information

No response

jkasten80 commented 1 year ago

image

jrasm91 commented 1 year ago

The image doesn't have city/state/country information in it directly. We use a reverse geocoding library that relies on data dumps from geonames to do a reverse lookup from the gps coordinates. The accuracy of that information is tied to the precision the library is configured to use, combined with the accuracy of the data. Immich itself is not going to be able to do much in terms of accuracy.

See:

dominikheinz commented 1 year ago

We can adjust the precision in the Dockerfile, but what is the default? (By default the precision env var is commented out). I'm experiencing the same issue, - and it is not because the town is to small.

jrasm91 commented 1 year ago

You can turn on verbose logging and reupload the picture in question to see what data we're getting back. This is pretty much all the code:

https://github.com/immich-app/immich/blob/main/server/libs/infra/src/repositories/geocoding.repository.ts#L45-L56

jkasten80 commented 1 year ago

I checked the coordinates of the pictures with the geonames database and the coordinates and names are correct in the database but the names in the pictures are wrong.

image image

jrasm91 commented 1 year ago

Have you tried reuploading the photo with verbose logging?

jkasten80 commented 1 year ago

How can I do this? When I try to delete the pic it will be deleted also from the phone.

jrasm91 commented 1 year ago

Not 100% sure about how to do it on the phone. You can delete in from the web though.

jkasten80 commented 1 year ago

Ahh sure... I'll try it and give feedback. I'll check some more pics and no Pic have the correct geolocation name. The map is always correct.

jkasten80 commented 1 year ago

I reuploaded the pic with verbose log on, which logs do you need now? Which container?

jkasten80 commented 1 year ago

I found it.. in the logs its also "Kronshagen" which is wrong. do you need anything special from the logs?

jrasm91 commented 1 year ago

Let me see if I can investigate.

jkasten80 commented 1 year ago

[Nest] 1 - 06/07/2023, 12:48:52 PM DEBUG [GeocodingRepository] Normalized: {"country":"Germany","state":"Schleswig-Holstein","city":"Kronshagen"} [Nest] 1 - 06/07/2023, 12:48:53 PM DEBUG [GeocodingRepository] Request: 54.3158861111111,10.0692583333333 [Nest] 1 - 06/07/2023, 12:48:53 PM VERBOSE [GeocodingRepository] Raw: [ { "geoNameId": "2883647", "name": "Kronshagen", "asciiName": "Kronshagen", "alternateNames": "Kronshagen,Kronskhagen,ke long sha gen,Кронсхаген,克龙沙根", "latitude": "54.33333", "longitude": "10.08333", "featureClass": "P", "featureCode": "PPLA4", "countryCode": "DE", "cc2": null, "admin1Code": { "name": "Schleswig-Holstein", "asciiName": "Schleswig-Holstein", "geoNameId": "2838632" }, "admin2Code": "00", "admin3Code": "01058", "admin4Code": "01058092", "population": "11856", "elevation": null, "dem": "23", "timezone": "Europe/Berlin", "modificationDate": "2011-07-31", "distance": 2.143599727736647 }

Here are some logs .. What does the modification date say?

jrasm91 commented 1 year ago

It's not going to match Hasseldieksdamm because it doesn't show up in the cities1000 dump (it doesn't have a population and the cities files we're using only includes cities with 1000+ population).

image

jkasten80 commented 1 year ago

Ok, the way to fix this would be to put 1000 under population? Thats a little bit strange. There are a lot parts of Kiel that could be under 1000 People. Kronshagen is defintiv wrong, Kiel would be ok because Hasseldieksdamm is a part of Kiel

jrasm91 commented 1 year ago

Immich, a photo management tool, has implemented this feature, reverse geocoding, by using data provided from http://www.geonames.org/. For questions about the accuracy of the data, you will have to consult geonames directly. For requests to update the data we're using cities1000.txt, you will also have to consult geonames. Obviously, the data is not going to be perfect and I'm not sure where they get the information from for this specific region. The data is provided for free, so it's the best we can do at the moment. I think, while technically not the closest populated city, Kronshagen is the closest known city and happens to be only one to two miles away from the original gps point.

jkasten80 commented 1 year ago

But Kronshagen is not Kiel, and Hasseldieksdamm is a part of Kiel. Kiel has 250k population. Ill try to fix this over geonames

jkasten80 commented 1 year ago

Maybe for future updates on your photo management software, could you integrate google api as an alternativ for people who want this (to use an own key)

jrasm91 commented 1 year ago

Feel free to open a feature request for an alternative reverse geocoding implementation (google maps). We could definitely add a setting for a google maps key and use that, which is probably much more accurate. The implementation we use doesn't take city/country boundaries into consideration. It's just the distance between the point of the image and the gps locations of the cities in the dump from geonames.

jkasten80 commented 1 year ago

That would be great! Your software is awesome and i like to use it more. Ill try to put a feature request .