search

friendlyFriend4000 / prometheus-immich-exporter

A prometheus exporter for immich written in Python. Docker image
https://github.com/immich-app/immich#demo
GNU General Public License v3.0
25 stars 9 forks source link

Error on container start #10

Closed theAlevan closed 5 months ago

theAlevan commented 8 months ago

When I start the container, I get this error message:

{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Exporter is starting up"}
{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Found immich up and running at immich_server:3001."}
{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Attempting to connect to immich"}
{"asctime": "2024-03-07 17:28:28", "levelname": "INFO", "message": "Exporter v1.0.9"}
{"asctime": "2024-03-07 17:28:28", "levelname": "INFO", "message": "Success."}
Traceback (most recent call last):
  File "/usr/bin/immich_exporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 385, in main
    REGISTRY.register(ImmichMetricsCollector(config))
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 40, in register
    names = self._get_names(collector)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 80, in _get_names
    for metric in desc_func():
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 25, in collect
    metrics = self.get_immich_metrics()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 46, in get_immich_metrics
    metrics.extend(self.get_immich_server_info())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 194, in get_immich_server_info
    "value": (response_server_info.json()["diskAvailableRaw"]),
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'diskAvailableRaw'

While it seems to succesfully connect to my Immich instance, it stops at this 'diskAvaliableRaw' error. Immich is on version v1.97.0.

This is part of my docker compose:

  prometheus-immich-exporter:
    container_name: immich_exporter
    image: friendlyfriend/prometheus-immich-exporter:latest
    environment:
      - IMMICH_PORT=3001
      - IMMICH_HOST=immich_server
      - IMMICH_API_TOKEN="myapitoken"
      - EXPORTER_PORT=8080
    networks:
      - internal
      - monitoring
    restart: unless-stopped
friendlyFriend4000 commented 8 months ago

When I start the container, I get this error message:

{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Exporter is starting up"}
{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Found immich up and running at immich_server:3001."}
{"asctime": "2024-03-07 17:28:27", "levelname": "INFO", "message": "Attempting to connect to immich"}
{"asctime": "2024-03-07 17:28:28", "levelname": "INFO", "message": "Exporter v1.0.9"}
{"asctime": "2024-03-07 17:28:28", "levelname": "INFO", "message": "Success."}
Traceback (most recent call last):
  File "/usr/bin/immich_exporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 385, in main
    REGISTRY.register(ImmichMetricsCollector(config))
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 40, in register
    names = self._get_names(collector)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 80, in _get_names
    for metric in desc_func():
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 25, in collect
    metrics = self.get_immich_metrics()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 46, in get_immich_metrics
    metrics.extend(self.get_immich_server_info())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 194, in get_immich_server_info
    "value": (response_server_info.json()["diskAvailableRaw"]),
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'diskAvailableRaw'

While it seems to succesfully connect to my Immich instance, it stops at this 'diskAvaliableRaw' error. Immich is on version v1.97.0.

This is part of my docker compose:

  prometheus-immich-exporter:
    container_name: immich_exporter
    image: friendlyfriend/prometheus-immich-exporter:latest
    environment:
      - IMMICH_PORT=3001
      - IMMICH_HOST=immich_server
      - IMMICH_API_TOKEN="myapitoken"
      - EXPORTER_PORT=8080
    networks:
      - internal
      - monitoring
    restart: unless-stopped

It's able to ping your immich instance, so it's online. However it fails to pull any data with the immich api token. Can you create a new token and try it again?

adityakun1992 commented 6 months ago

Hi, I just tried it and see a similar issue for a different key:

{"asctime": "2024-05-01 05:14:20", "levelname": "INFO", "message": "Exporter is starting up"}
{"asctime": "2024-05-01 05:14:20", "levelname": "INFO", "message": "Found immich up and running at 192.168.1.43:2283."}
{"asctime": "2024-05-01 05:14:20", "levelname": "INFO", "message": "Attempting to connect to immich"}
{"asctime": "2024-05-01 05:14:21", "levelname": "INFO", "message": "Exporter v1.0.9"}
{"asctime": "2024-05-01 05:14:21", "levelname": "INFO", "message": "Success."}
{"asctime": "2024-05-01 05:14:21", "levelname": "ERROR", "message": "Is the Immich api token valid? Traceback:KeyError: 'usageByUser': "}
Traceback (most recent call last):
  File "/usr/bin/immich_exporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 385, in main
    REGISTRY.register(ImmichMetricsCollector(config))
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 40, in register
    names = self._get_names(collector)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 80, in _get_names
    for metric in desc_func():
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 25, in collect
    metrics = self.get_immich_metrics()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 47, in get_immich_metrics
    metrics.extend(self.get_immich_users_stat)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 130, in get_immich_users_stat
    user_data = response_user_stats.json()["usageByUser"]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'usageByUser'
{"asctime": "2024-05-01 05:14:32", "levelname": "INFO", "message": "Exporter is starting up"}
{"asctime": "2024-05-01 05:14:32", "levelname": "INFO", "message": "Found immich up and running at 192.168.1.43:2283."}
{"asctime": "2024-05-01 05:14:32", "levelname": "INFO", "message": "Attempting to connect to immich"}
{"asctime": "2024-05-01 05:14:33", "levelname": "INFO", "message": "Exporter v1.0.9"}
{"asctime": "2024-05-01 05:14:33", "levelname": "INFO", "message": "Success."}
{"asctime": "2024-05-01 05:14:33", "levelname": "ERROR", "message": "Is the Immich api token valid? Traceback:KeyError: 'usageByUser': "}
Traceback (most recent call last):
  File "/usr/bin/immich_exporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 385, in main
    REGISTRY.register(ImmichMetricsCollector(config))
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 40, in register
    names = self._get_names(collector)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/prometheus_client/registry.py", line 80, in _get_names
    for metric in desc_func():
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 25, in collect
    metrics = self.get_immich_metrics()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 47, in get_immich_metrics
    metrics.extend(self.get_immich_users_stat)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/immich_exporter/exporter.py", line 130, in get_immich_users_stat
    user_data = response_user_stats.json()["usageByUser"]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'usageByUser'

I tried two different tokens and verified both are valid. If it matters,

Immich Version: 1.101.0 OS: Unraid 6.12.8 Environment: Docker (Unraid Community Apps)

friendlyFriend4000 commented 6 months ago

The usageByUser api endpoint is the first one being called in the program, basically it fails to connect to your immich instance but it sees it. It works with the following docker command: docker run -e IMMICH_PORT=2283 -e IMMICH_HOST=192.168.178.2 -e IMMICH_API_TOKEN=3asdadsdsatoekasdasadsaasfsaf -p 8000:8000 friendlyfriend/prometheus-immich-exporter

use the command above and fill out your values and I am sure it will work.

On a side note I am very close to releasing a reworked webapp version of this exporter, that works without grafana or prometheus, so if it doesn't work you may wanna try that out.

DinStamou commented 5 months ago

I get the same with @adityakun1992 , same setup and same keyErrors

friendlyFriend4000 commented 5 months ago

I get the same with @adityakun1992 , same setup and same keyErrors

the key error is being thrown because it's the first key that's being queried. It's actually a query issue. your immich server is online, but for some reason it cannot connect to it. Can you run me through your setup in detail so i can try to recreate this error.

dhiorns15 commented 5 months ago

Howdy everyone.

It appears that in file /immich_exporter/exporter.py on line 181 you are trying to query /api/server-info. This endpoint has moved to /api/server-info/storage. Updating that endpoint will allow the plugin to function again.

This may have moved this in newer versions of the API.

Docs: https://immich.app/docs/api/get-storage

3rdStng commented 5 months ago

Any chance of getting the docker image updated to support the change?

friendlyFriend4000 commented 5 months ago

Howdy everyone.

It appears that in file /immich_exporter/exporter.py on line 181 you are trying to query /api/server-info. This endpoint has moved to /api/server-info/storage. Updating that endpoint will allow the plugin to function again.

This may have moved this in newer versions of the API.

Docs: https://immich.app/docs/api/get-storage

thank you for pointing that out, my very own immich exporter isntance wasnt working and i haven't noticed. i ll push out a fix today or tomorrow.

friendlyFriend4000 commented 5 months ago

Thank you for your contribution. I appreciate it.

alex-007 commented 5 months ago

@friendlyFriend4000 No, thank you for your original work! BTW I think, you can close this issue now :)