immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
38.99k stars 1.84k forks source link

OpenVino: facial detection is broken with 1.99.0 #8226

Open LeoAdL opened 3 months ago

LeoAdL commented 3 months ago

The bug

When I try launching the face detection, whatever model I use, I get the following error:

immich_machine_learning  | [03/23/24 22:46:05] INFO     Setting 'buffalo_l' execution providers to         
immich_machine_learning  |                              ['OpenVINOExecutionProvider',                      
immich_machine_learning  |                              'CPUExecutionProvider'], in descending order of    
immich_machine_learning  |                              preference                                         
immich_machine_learning  | [03/23/24 22:46:05] INFO     Loading facial recognition model 'buffalo_l' to    
immich_machine_learning  |                              memory                                             
immich_machine_learning  | [03/23/24 22:46:08] ERROR    Exception in ASGI application                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:118 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   115 │                                         │
immich_machine_learning  |                              │   116 │   model = await load(await model_cache. │
immich_machine_learning  |                              │       ttl=settings.model_ttl, **kwargs))        │
immich_machine_learning  |                              │   117 │   model.configure(**kwargs)             │
immich_machine_learning  |                              │ ❱ 118 │   outputs = await run(model.predict, in │
immich_machine_learning  |                              │   119 │   return ORJSONResponse(outputs)        │
immich_machine_learning  |                              │   120                                           │
immich_machine_learning  |                              │   121                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:125 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   122 async def run(func: Callable[..., Any], i │
immich_machine_learning  |                              │   123 │   if thread_pool is None:               │
immich_machine_learning  |                              │   124 │   │   return func(inputs)               │
immich_machine_learning  |                              │ ❱ 125 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   126                                           │
immich_machine_learning  |                              │   127                                           │
immich_machine_learning  |                              │   128 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:59 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    56 │   │   self.load()                       │
immich_machine_learning  |                              │    57 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    58 │   │   │   self.configure(**model_kwargs │
immich_machine_learning  |                              │ ❱  59 │   │   return self._predict(inputs)      │
immich_machine_learning  |                              │    60 │                                         │
immich_machine_learning  |                              │    61 │   @abstractmethod                       │
immich_machine_learning  |                              │    62 │   def _predict(self, inputs: Any) -> An │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/facial_recognition.py:49 in │
immich_machine_learning  |                              │ _predict                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   46 │   │   else:                              │
immich_machine_learning  |                              │   47 │   │   │   decoded_image = image          │
immich_machine_learning  |                              │   48 │   │   assert is_ndarray(decoded_image, n │
immich_machine_learning  |                              │ ❱ 49 │   │   bboxes, kpss = self.det_model.dete │
immich_machine_learning  |                              │   50 │   │   if bboxes.size == 0:               │
immich_machine_learning  |                              │   51 │   │   │   return []                      │
immich_machine_learning  |                              │   52 │   │   assert is_ndarray(kpss, np.float32 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:224 in detect       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   221 │   │   det_img = np.zeros( (input_size[1 │
immich_machine_learning  |                              │   222 │   │   det_img[:new_height, :new_width,  │
immich_machine_learning  |                              │   223 │   │                                     │
immich_machine_learning  |                              │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
immich_machine_learning  |                              │   225 │   │                                     │
immich_machine_learning  |                              │   226 │   │   scores = np.vstack(scores_list)   │
immich_machine_learning  |                              │   227 │   │   scores_ravel = scores.ravel()     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:152 in forward      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   149 │   │   kpss_list = []                    │
immich_machine_learning  |                              │   150 │   │   input_size = tuple(img.shape[0:2] │
immich_machine_learning  |                              │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
immich_machine_learning  |                              │       (self.input_mean, self.input_mean, self.i │
immich_machine_learning  |                              │ ❱ 152 │   │   net_outs = self.session.run(self. │
immich_machine_learning  |                              │   153 │   │                                     │
immich_machine_learning  |                              │   154 │   │   input_height = blob.shape[2]      │
immich_machine_learning  |                              │   155 │   │   input_width = blob.shape[3]       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:22 │
immich_machine_learning  |                              │ 0 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    217 │   │   if not output_names:             │
immich_machine_learning  |                              │    218 │   │   │   output_names = [output.name  │
immich_machine_learning  |                              │    219 │   │   try:                             │
immich_machine_learning  |                              │ ❱  220 │   │   │   return self._sess.run(output │
immich_machine_learning  |                              │    221 │   │   except C.EPFail as err:          │
immich_machine_learning  |                              │    222 │   │   │   if self._enable_fallback:    │
immich_machine_learning  |                              │    223 │   │   │   │   print(f"EP Error: {err!s │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              RuntimeException: [ONNXRuntimeError] : 6 :         
immich_machine_learning  |                              RUNTIME_EXCEPTION : Encountered unknown exception  
immich_machine_learning  |                              in Run()                                           
immich_microservices     | [Nest] 7  - 03/23/2024, 10:46:08 PM   ERROR [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     | [Nest] 7  - 03/23/2024, 10:46:08 PM   ERROR [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     |     at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:23:19)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async PersonService.handleDetectFaces (/usr/src/app/dist/domain/person/person.service.js:248:23)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:137:36
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices     | [Nest] 7  - 03/23/2024, 10:46:08 PM   ERROR [JobService] Object:
immich_microservices     | {
immich_microservices     |   "id": "9e1d4bbf-84c2-40dd-9aec-c913e5a1a662"
immich_microservices     | }
immich_microservices     | 

Regular Smart Search proceeds without issue.

The OS that Immich Server is running on

Proxmox 8.1 (6.5 Linux Kernel)

Version of Immich Server

1.99.0

Version of Immich Mobile App

1.99.0

Platform with the issue

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mount/op11:/usr/src/app/external
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends: 
     file: hwaccel.transcoding.yml 
     service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mount/op11:/usr/src/app/external
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:main-openvino
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=(redacted)

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. docker compose up
2. --> Jobs 
3. --> Click Face Detection: All
...

Additional information

My processor is an Intel N100. Previously to 1.99.0, the face detection was working, but I had issues with the smart search, so I guess it's hard to get all of it with OpenVino haha.

Thank you for the gigantic work up to now! Leo

mertalev commented 3 months ago

Can you check if setting the concurrency to 1 and restarting helps? And if that doesn't work, deleting the model cache and restarting?

LeoAdL commented 3 months ago

Setting the concurrency to 1, I almost get immediately a similar error:

immich_redis             | 1:M 24 Mar 2024 00:46:41.343 * Background saving terminated with success
immich_machine_learning  | [03/24/24 00:46:43] INFO     Setting 'buffalo_l' execution providers to         
immich_machine_learning  |                              ['OpenVINOExecutionProvider',                      
immich_machine_learning  |                              'CPUExecutionProvider'], in descending order of    
immich_machine_learning  |                              preference                                         
immich_machine_learning  | [03/24/24 00:46:43] INFO     Loading facial recognition model 'buffalo_l' to    
immich_machine_learning  |                              memory                                             
immich_machine_learning  | [03/24/24 00:46:45] ERROR    Exception in ASGI application                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:118 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   115 │                                         │
immich_machine_learning  |                              │   116 │   model = await load(await model_cache. │
immich_machine_learning  |                              │       ttl=settings.model_ttl, **kwargs))        │
immich_machine_learning  |                              │   117 │   model.configure(**kwargs)             │
immich_machine_learning  |                              │ ❱ 118 │   outputs = await run(model.predict, in │
immich_machine_learning  |                              │   119 │   return ORJSONResponse(outputs)        │
immich_machine_learning  |                              │   120                                           │
immich_machine_learning  |                              │   121                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:125 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   122 async def run(func: Callable[..., Any], i │
immich_machine_learning  |                              │   123 │   if thread_pool is None:               │
immich_machine_learning  |                              │   124 │   │   return func(inputs)               │
immich_machine_learning  |                              │ ❱ 125 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   126                                           │
immich_machine_learning  |                              │   127                                           │
immich_machine_learning  |                              │   128 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:59 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    56 │   │   self.load()                       │
immich_machine_learning  |                              │    57 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    58 │   │   │   self.configure(**model_kwargs │
immich_machine_learning  |                              │ ❱  59 │   │   return self._predict(inputs)      │
immich_machine_learning  |                              │    60 │                                         │
immich_machine_learning  |                              │    61 │   @abstractmethod                       │
immich_machine_learning  |                              │    62 │   def _predict(self, inputs: Any) -> An │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/facial_recognition.py:49 in │
immich_machine_learning  |                              │ _predict                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   46 │   │   else:                              │
immich_machine_learning  |                              │   47 │   │   │   decoded_image = image          │
immich_machine_learning  |                              │   48 │   │   assert is_ndarray(decoded_image, n │
immich_machine_learning  |                              │ ❱ 49 │   │   bboxes, kpss = self.det_model.dete │
immich_machine_learning  |                              │   50 │   │   if bboxes.size == 0:               │
immich_machine_learning  |                              │   51 │   │   │   return []                      │
immich_machine_learning  |                              │   52 │   │   assert is_ndarray(kpss, np.float32 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:224 in detect       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   221 │   │   det_img = np.zeros( (input_size[1 │
immich_machine_learning  |                              │   222 │   │   det_img[:new_height, :new_width,  │
immich_machine_learning  |                              │   223 │   │                                     │
immich_machine_learning  |                              │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
immich_machine_learning  |                              │   225 │   │                                     │
immich_machine_learning  |                              │   226 │   │   scores = np.vstack(scores_list)   │
immich_machine_learning  |                              │   227 │   │   scores_ravel = scores.ravel()     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:152 in forward      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   149 │   │   kpss_list = []                    │
immich_machine_learning  |                              │   150 │   │   input_size = tuple(img.shape[0:2] │
immich_machine_learning  |                              │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
immich_machine_learning  |                              │       (self.input_mean, self.input_mean, self.i │
immich_machine_learning  |                              │ ❱ 152 │   │   net_outs = self.session.run(self. │
immich_machine_learning  |                              │   153 │   │                                     │
immich_machine_learning  |                              │   154 │   │   input_height = blob.shape[2]      │
immich_machine_learning  |                              │   155 │   │   input_width = blob.shape[3]       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:22 │
immich_machine_learning  |                              │ 0 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    217 │   │   if not output_names:             │
immich_machine_learning  |                              │    218 │   │   │   output_names = [output.name  │
immich_machine_learning  |                              │    219 │   │   try:                             │
immich_machine_learning  |                              │ ❱  220 │   │   │   return self._sess.run(output │
immich_machine_learning  |                              │    221 │   │   except C.EPFail as err:          │
immich_machine_learning  |                              │    222 │   │   │   if self._enable_fallback:    │
immich_machine_learning  |                              │    223 │   │   │   │   print(f"EP Error: {err!s │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              RuntimeException: [ONNXRuntimeError] : 6 :         
immich_machine_learning  |                              RUNTIME_EXCEPTION : Encountered unknown exception  
immich_machine_learning  |                              in Run()                                           
immich_microservices     | [Nest] 7  - 03/24/2024, 12:46:45 AM   ERROR [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     | [Nest] 7  - 03/24/2024, 12:46:45 AM   ERROR [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     |     at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:23:19)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async PersonService.handleDetectFaces (/usr/src/app/dist/domain/person/person.service.js:248:23)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:137:36
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices     | [Nest] 7  - 03/24/2024, 12:46:45 AM   ERROR [JobService] Object:
immich_microservices     | {
immich_microservices     |   "id": "9e1d4bbf-84c2-40dd-9aec-c913e5a1a662"
immich_microservices     | }
immich_microservices     | 

Deleting the model cache did not help neither.

mertalev commented 3 months ago

Can you try again with these env variables set?

ORT_OPENVINO_ENABLE_CI_LOG=1
ORT_OPENVINO_ENABLE_DEBUG=1
OPENVINO_LOG_LEVEL=5
LOG_LEVEL=debug

This will help get more info on what's causing the error.

edjacob25 commented 3 months ago

I have the same issue with the N100 too. I tried changing the model to see if that was a workaround but I'm still getting the issue. After adding those variables I got this:

Complete logs ``` In the OpenVINO EP Model is fully supported on OpenVINO CreateNgraphFunc In the OpenVINO EP Model is fully supported on OpenVINO CreateNgraphFunc [03/24/24 06:07:37] ERROR Exception in ASGI application ╭─────── Traceback (most recent call last) ───────╮ │ /usr/src/app/main.py:118 in predict │ │ │ │ 115 │ │ │ 116 │ model = await load(await model_cache. │ │ ttl=settings.model_ttl, **kwargs)) │ │ 117 │ model.configure(**kwargs) │ │ ❱ 118 │ outputs = await run(model.predict, in │ │ 119 │ return ORJSONResponse(outputs) │ │ 120 │ │ 121 │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ image = UploadFile(filename='blob', │ │ │ │ size=212656, │ │ │ │ headers=Headers({'content-dis… │ │ │ │ 'form-data; name="image"; │ │ │ │ filename="blob"', │ │ │ │ 'content-type': │ │ │ │ 'application/octet-stream'})) │ │ │ │ inputs = b'\xff\xd8\xff\xe2\x01\xf0ICC… │ │ │ │ \x00\x00mntrRGB XYZ │ │ │ │ \x07\xe2\x00\x03\x00\x14\x00\… │ │ │ │ kwargs = { │ │ │ │ │ 'minScore': 0.7, │ │ │ │ │ 'maxDistance': 0.5, │ │ │ │ │ 'minFaces': 3 │ │ │ │ } │ │ │ │ model = │ │ │ │ model_name = 'antelopev2' │ │ │ │ model_type = │ │ │ │ options = '{"minScore":0.7,"maxDistance… │ │ │ │ text = None │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /usr/src/app/main.py:125 in run │ │ │ │ 122 async def run(func: Callable[..., Any], i │ │ 123 │ if thread_pool is None: │ │ 124 │ │ return func(inputs) │ │ ❱ 125 │ return await asyncio.get_running_loop │ │ 126 │ │ 127 │ │ 128 async def load(model: InferenceModel) -> │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ func = > │ │ │ │ inputs = b'\xff\xd8\xff\xe2\x01\xf0ICC_PRO… │ │ │ │ \x00\x00mntrRGB XYZ │ │ │ │ \x07\xe2\x00\x03\x00\x14\x00\t\x0… │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /usr/lib/python3.10/concurrent/futures/thread.p │ │ y:58 in run │ │ │ │ /usr/src/app/models/base.py:59 in predict │ │ │ │ 56 │ │ self.load() │ │ 57 │ │ if model_kwargs: │ │ 58 │ │ │ self.configure(**model_kwargs │ │ ❱ 59 │ │ return self._predict(inputs) │ │ 60 │ │ │ 61 │ @abstractmethod │ │ 62 │ def _predict(self, inputs: Any) -> An │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ inputs = b'\xff\xd8\xff\xe2\x01\xf0I… │ │ │ │ \x00\x00mntrRGB XYZ │ │ │ │ \x07\xe2\x00\x03\x00\x14\x0… │ │ │ │ model_kwargs = {} │ │ │ │ self = │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /usr/src/app/models/facial_recognition.py:49 in │ │ _predict │ │ │ │ 46 │ │ else: │ │ 47 │ │ │ decoded_image = image │ │ 48 │ │ assert is_ndarray(decoded_image, n │ │ ❱ 49 │ │ bboxes, kpss = self.det_model.dete │ │ 50 │ │ if bboxes.size == 0: │ │ 51 │ │ │ return [] │ │ 52 │ │ assert is_ndarray(kpss, np.float32 │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ decoded_image = array([[[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ ..., │ │ │ │ │ │ │ │ │ │ [[ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4]], │ │ │ │ │ │ │ │ │ │ [[ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3]], │ │ │ │ │ │ │ │ │ │ [[ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3]]], │ │ │ │ dtype=uint8) │ │ │ │ image = b'\xff\xd8\xff\xe2\x01\xf0… │ │ │ │ \x00\x00mntrRGB XYZ │ │ │ │ \x07\xe2\x00\x03\x00\x14\x… │ │ │ │ self = │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /opt/venv/lib/python3.10/site-packages/insightf │ │ ace/model_zoo/retinaface.py:224 in detect │ │ │ │ 221 │ │ det_img = np.zeros( (input_size[1 │ │ 222 │ │ det_img[:new_height, :new_width, │ │ 223 │ │ │ │ ❱ 224 │ │ scores_list, bboxes_list, kpss_li │ │ 225 │ │ │ │ 226 │ │ scores = np.vstack(scores_list) │ │ 227 │ │ scores_ravel = scores.ravel() │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ det_img = array([[[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ ..., │ │ │ │ │ │ │ │ │ │ [[ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]]], │ │ │ │ dtype=uint8) │ │ │ │ det_scale = 0.3333333333333333 │ │ │ │ im_ratio = 1.3333333333333333 │ │ │ │ img = array([[[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ ..., │ │ │ │ │ │ │ │ │ │ [[ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4]], │ │ │ │ │ │ │ │ │ │ [[ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3]], │ │ │ │ │ │ │ │ │ │ [[ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3]]], │ │ │ │ dtype=uint8) │ │ │ │ input_size = (640, 640) │ │ │ │ max_num = 0 │ │ │ │ metric = 'default' │ │ │ │ model_ratio = 1.0 │ │ │ │ new_height = 640 │ │ │ │ new_width = 480 │ │ │ │ resized_img = array([[[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116], │ │ │ │ │ │ [227, 154, 116]], │ │ │ │ │ │ │ │ │ │ ..., │ │ │ │ │ │ │ │ │ │ [[ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 11, 11, 5], │ │ │ │ │ │ [ 11, 11, 5], │ │ │ │ │ │ [ 11, 11, 5]], │ │ │ │ │ │ │ │ │ │ [[ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4], │ │ │ │ │ │ [ 10, 10, 4]], │ │ │ │ │ │ │ │ │ │ [[ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3], │ │ │ │ │ │ [ 9, 9, 3]]], │ │ │ │ dtype=uint8) │ │ │ │ self = │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /opt/venv/lib/python3.10/site-packages/insightf │ │ ace/model_zoo/retinaface.py:152 in forward │ │ │ │ 149 │ │ kpss_list = [] │ │ 150 │ │ input_size = tuple(img.shape[0:2] │ │ 151 │ │ blob = cv2.dnn.blobFromImage(img, │ │ (self.input_mean, self.input_mean, self.i │ │ ❱ 152 │ │ net_outs = self.session.run(self. │ │ 153 │ │ │ │ 154 │ │ input_height = blob.shape[2] │ │ 155 │ │ input_width = blob.shape[3] │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ bboxes_list = [] │ │ │ │ blob = array([[[[-0.18359375, │ │ │ │ -0.18359375, -0.18359375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.17578125, │ │ │ │ -0.17578125, -0.17578125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.17578125, │ │ │ │ -0.17578125, -0.17578125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.83984375, │ │ │ │ -0.83984375, -0.83984375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.82421875, │ │ │ │ -0.82421875, -0.82421875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.83203125, │ │ │ │ -0.83203125, -0.83203125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]], │ │ │ │ │ │ │ │ │ │ │ │ [[ 0.09765625, │ │ │ │ 0.09765625, 0.09765625, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.10546875, │ │ │ │ 0.10546875, 0.10546875, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.10546875, │ │ │ │ 0.10546875, 0.10546875, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.78515625, │ │ │ │ -0.78515625, -0.78515625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.76953125, │ │ │ │ -0.76953125, -0.76953125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.77734375, │ │ │ │ -0.77734375, -0.77734375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]], │ │ │ │ │ │ │ │ │ │ │ │ [[ 0.69140625, │ │ │ │ 0.69140625, 0.69140625, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.69921875, │ │ │ │ 0.69921875, 0.69921875, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.69921875, │ │ │ │ 0.69921875, 0.69921875, ..., │ │ │ │ -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.64453125, │ │ │ │ -0.64453125, -0.64453125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.62890625, │ │ │ │ -0.62890625, -0.62890625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.63671875, │ │ │ │ -0.63671875, -0.63671875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]]]], │ │ │ │ dtype=float32) │ │ │ │ img = array([[[216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ [216, 140, 104], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ [217, 141, 105], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ ..., │ │ │ │ │ │ │ │ │ │ [[ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ [ 45, 27, 20], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ [ 47, 29, 22], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]], │ │ │ │ │ │ │ │ │ │ [[ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ [ 46, 28, 21], │ │ │ │ │ │ ..., │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0], │ │ │ │ │ │ [ 0, 0, 0]]], │ │ │ │ dtype=uint8) │ │ │ │ input_size = (640, 640) │ │ │ │ kpss_list = [] │ │ │ │ scores_list = [] │ │ │ │ self = │ │ │ │ threshold = 0.7 │ │ │ ╰─────────────────────────────────────────────╯ │ │ │ │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ │ ime/capi/onnxruntime_inference_collection.py:22 │ │ 0 in run │ │ │ │ 217 │ │ if not output_names: │ │ 218 │ │ │ output_names = [output.name │ │ 219 │ │ try: │ │ ❱ 220 │ │ │ return self._sess.run(output │ │ 221 │ │ except C.EPFail as err: │ │ 222 │ │ │ if self._enable_fallback: │ │ 223 │ │ │ │ print(f"EP Error: {err!s │ │ │ │ ╭────────────────── locals ───────────────────╮ │ │ │ input_feed = { │ │ │ │ │ 'input.1': │ │ │ │ array([[[[-0.18359375, │ │ │ │ -0.18359375, -0.18359375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.17578125, │ │ │ │ -0.17578125, -0.17578125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.17578125, │ │ │ │ -0.17578125, -0.17578125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.83984375, │ │ │ │ -0.83984375, -0.83984375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.82421875, │ │ │ │ -0.82421875, -0.82421875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.83203125, │ │ │ │ -0.83203125, -0.83203125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]], │ │ │ │ │ │ │ │ │ │ │ │ [[ 0.09765625, │ │ │ │ 0.09765625, 0.09765625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.10546875, │ │ │ │ 0.10546875, 0.10546875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.10546875, │ │ │ │ 0.10546875, 0.10546875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.78515625, │ │ │ │ -0.78515625, -0.78515625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.76953125, │ │ │ │ -0.76953125, -0.76953125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.77734375, │ │ │ │ -0.77734375, -0.77734375, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]], │ │ │ │ │ │ │ │ │ │ │ │ [[ 0.69140625, │ │ │ │ 0.69140625, 0.69140625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.69921875, │ │ │ │ 0.69921875, 0.69921875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [ 0.69921875, │ │ │ │ 0.69921875, 0.69921875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ ..., │ │ │ │ │ │ [-0.64453125, │ │ │ │ -0.64453125, -0.64453125, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.62890625, │ │ │ │ -0.62890625, -0.62890625, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375], │ │ │ │ │ │ [-0.63671875, │ │ │ │ -0.63671875, -0.63671875, │ │ │ │ ..., -0.99609375, │ │ │ │ │ │ -0.99609375, │ │ │ │ -0.99609375]]]], │ │ │ │ dtype=float32) │ │ │ │ } │ │ │ │ output_names = [ │ │ │ │ │ '448', │ │ │ │ │ '471', │ │ │ │ │ '494', │ │ │ │ │ '451', │ │ │ │ │ '474', │ │ │ │ │ '497', │ │ │ │ │ '454', │ │ │ │ │ '477', │ │ │ │ │ '500' │ │ │ │ ] │ │ │ │ run_options = None │ │ │ │ self = │ │ │ ╰─────────────────────────────────────────────╯ │ ╰─────────────────────────────────────────────────╯ RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Encountered unknown exception in Run() ```

Hope this helps!

LeoAdL commented 3 months ago

Here are my logs, I think very similar:

immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Available ORT providers:                           
immich_machine_learning  |                              {'OpenVINOExecutionProvider',                      
immich_machine_learning  |                              'CPUExecutionProvider'}                            
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Available OpenVINO devices: ['CPU', 'GPU']         
immich_machine_learning  | [03/24/24 16:17:42] INFO     Setting 'buffalo_l' execution providers to         
immich_machine_learning  |                              ['OpenVINOExecutionProvider',                      
immich_machine_learning  |                              'CPUExecutionProvider'], in descending order of    
immich_machine_learning  |                              preference                                         
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Setting execution provider options to              
immich_machine_learning  |                              [{'device_type': 'GPU_FP32', 'cache_dir':          
immich_machine_learning  |                              '/cache/facial-recognition/buffalo_l/openvino'},   
immich_machine_learning  |                              {'arena_extend_strategy': 'kSameAsRequested'}]     
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Setting execution_mode to ORT_SEQUENTIAL           
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Setting inter_op_num_threads to 0                  
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Setting intra_op_num_threads to 0                  
immich_machine_learning  | [03/24/24 16:17:42] DEBUG    Setting preferred runtime to onnx                  
immich_machine_learning  | [03/24/24 16:17:42] INFO     Loading facial recognition model 'buffalo_l' to    
immich_machine_learning  |                              memory                                             
immich_machine_learning  | In the OpenVINO EP
immich_machine_learning  | Model is fully supported on OpenVINO
immich_machine_learning  | CreateNgraphFunc
immich_machine_learning  | In the OpenVINO EP
immich_machine_learning  | Model is fully supported on OpenVINO
immich_machine_learning  | CreateNgraphFunc
immich_machine_learning  | [03/24/24 16:17:44] ERROR    Exception in ASGI application                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:118 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   115 │                                         │
immich_machine_learning  |                              │   116 │   model = await load(await model_cache. │
immich_machine_learning  |                              │       ttl=settings.model_ttl, **kwargs))        │
immich_machine_learning  |                              │   117 │   model.configure(**kwargs)             │
immich_machine_learning  |                              │ ❱ 118 │   outputs = await run(model.predict, in │
immich_machine_learning  |                              │   119 │   return ORJSONResponse(outputs)        │
immich_machine_learning  |                              │   120                                           │
immich_machine_learning  |                              │   121                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │      image = UploadFile(filename='blob',    │ │
immich_machine_learning  |                              │ │              size=591599,                   │ │
immich_machine_learning  |                              │ │              headers=Headers({'content-dis… │ │
immich_machine_learning  |                              │ │              'form-data; name="image";      │ │
immich_machine_learning  |                              │ │              filename="blob"',              │ │
immich_machine_learning  |                              │ │              'content-type':                │ │
immich_machine_learning  |                              │ │              'application/octet-stream'}))  │ │
immich_machine_learning  |                              │ │     inputs = b'\xff\xd8\xff\xe2\x01\xf0ICC… │ │
immich_machine_learning  |                              │ │              \x00\x00mntrRGB XYZ            │ │
immich_machine_learning  |                              │ │              \x07\xe2\x00\x03\x00\x14\x00\… │ │
immich_machine_learning  |                              │ │     kwargs = {                              │ │
immich_machine_learning  |                              │ │              │   'minScore': 0.7,           │ │
immich_machine_learning  |                              │ │              │   'maxDistance': 0.5,        │ │
immich_machine_learning  |                              │ │              │   'minFaces': 3              │ │
immich_machine_learning  |                              │ │              }                              │ │
immich_machine_learning  |                              │ │      model = <app.models.facial_recognitio… │ │
immich_machine_learning  |                              │ │              object at 0x7ab23c362fe0>      │ │
immich_machine_learning  |                              │ │ model_name = 'buffalo_l'                    │ │
immich_machine_learning  |                              │ │ model_type = <ModelType.FACIAL_RECOGNITION: │ │
immich_machine_learning  |                              │ │              'facial-recognition'>          │ │
immich_machine_learning  |                              │ │    options = '{"minScore":0.7,"maxDistance… │ │
immich_machine_learning  |                              │ │       text = None                           │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:125 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   122 async def run(func: Callable[..., Any], i │
immich_machine_learning  |                              │   123 │   if thread_pool is None:               │
immich_machine_learning  |                              │   124 │   │   return func(inputs)               │
immich_machine_learning  |                              │ ❱ 125 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   126                                           │
immich_machine_learning  |                              │   127                                           │
immich_machine_learning  |                              │   128 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │   func = <bound method                      │ │
immich_machine_learning  |                              │ │          InferenceModel.predict of          │ │
immich_machine_learning  |                              │ │          <app.models.facial_recognition.Fa… │ │
immich_machine_learning  |                              │ │          object at 0x7ab23c362fe0>>         │ │
immich_machine_learning  |                              │ │ inputs = b'\xff\xd8\xff\xe2\x01\xf0ICC_PRO… │ │
immich_machine_learning  |                              │ │          \x00\x00mntrRGB XYZ                │ │
immich_machine_learning  |                              │ │          \x07\xe2\x00\x03\x00\x14\x00\t\x0… │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:59 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    56 │   │   self.load()                       │
immich_machine_learning  |                              │    57 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    58 │   │   │   self.configure(**model_kwargs │
immich_machine_learning  |                              │ ❱  59 │   │   return self._predict(inputs)      │
immich_machine_learning  |                              │    60 │                                         │
immich_machine_learning  |                              │    61 │   @abstractmethod                       │
immich_machine_learning  |                              │    62 │   def _predict(self, inputs: Any) -> An │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │       inputs = b'\xff\xd8\xff\xe2\x01\xf0I… │ │
immich_machine_learning  |                              │ │                \x00\x00mntrRGB XYZ          │ │
immich_machine_learning  |                              │ │                \x07\xe2\x00\x03\x00\x14\x0… │ │
immich_machine_learning  |                              │ │ model_kwargs = {}                           │ │
immich_machine_learning  |                              │ │         self = <app.models.facial_recognit… │ │
immich_machine_learning  |                              │ │                object at 0x7ab23c362fe0>    │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/facial_recognition.py:49 in │
immich_machine_learning  |                              │ _predict                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   46 │   │   else:                              │
immich_machine_learning  |                              │   47 │   │   │   decoded_image = image          │
immich_machine_learning  |                              │   48 │   │   assert is_ndarray(decoded_image, n │
immich_machine_learning  |                              │ ❱ 49 │   │   bboxes, kpss = self.det_model.dete │
immich_machine_learning  |                              │   50 │   │   if bboxes.size == 0:               │
immich_machine_learning  |                              │   51 │   │   │   return []                      │
immich_machine_learning  |                              │   52 │   │   assert is_ndarray(kpss, np.float32 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │ decoded_image = array([[[  0,  17,  34],    │ │
immich_machine_learning  |                              │ │                 │   │   [  0,  17,  34],    │ │
immich_machine_learning  |                              │ │                 │   │   [  0,  18,  35],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [  8,   7,   9],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 14,  13,  15],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 19,  18,  20]],   │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      [[  0,  19,  36],    │ │
immich_machine_learning  |                              │ │                 │   │   [  1,  19,  36],    │ │
immich_machine_learning  |                              │ │                 │   │   [  2,  20,  37],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [  7,   6,   8],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 14,  13,  15],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 20,  19,  21]],   │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      [[  2,  22,  39],    │ │
immich_machine_learning  |                              │ │                 │   │   [  4,  22,  39],    │ │
immich_machine_learning  |                              │ │                 │   │   [  4,  22,  39],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [  5,   4,   6],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 14,  13,  15],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 22,  21,  23]],   │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      ...,                 │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      [[ 32,  56,  86],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 36,  60,  90],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 42,  66,  96],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [ 28,  54,  71],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 31,  57,  74],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 36,  62,  79]],   │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      [[ 37,  59,  87],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 43,  65,  93],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 53,  75, 103],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [ 27,  53,  70],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 28,  54,  71],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 32,  58,  75]],   │ │
immich_machine_learning  |                              │ │                 │                           │ │
immich_machine_learning  |                              │ │                 │      [[ 44,  66,  94],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 50,  72, 100],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 61,  83, 111],    │ │
immich_machine_learning  |                              │ │                 │   │   ...,                │ │
immich_machine_learning  |                              │ │                 │   │   [ 27,  53,  70],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 25,  51,  68],    │ │
immich_machine_learning  |                              │ │                 │   │   [ 27,  53,  70]]],  │ │
immich_machine_learning  |                              │ │                 dtype=uint8)                │ │
immich_machine_learning  |                              │ │         image = b'\xff\xd8\xff\xe2\x01\xf0… │ │
immich_machine_learning  |                              │ │                 \x00\x00mntrRGB XYZ         │ │
immich_machine_learning  |                              │ │                 \x07\xe2\x00\x03\x00\x14\x… │ │
immich_machine_learning  |                              │ │          self = <app.models.facial_recogni… │ │
immich_machine_learning  |                              │ │                 object at 0x7ab23c362fe0>   │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:224 in detect       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   221 │   │   det_img = np.zeros( (input_size[1 │
immich_machine_learning  |                              │   222 │   │   det_img[:new_height, :new_width,  │
immich_machine_learning  |                              │   223 │   │                                     │
immich_machine_learning  |                              │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
immich_machine_learning  |                              │   225 │   │                                     │
immich_machine_learning  |                              │   226 │   │   scores = np.vstack(scores_list)   │
immich_machine_learning  |                              │   227 │   │   scores_ravel = scores.ravel()     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │     det_img = array([[[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  13,  31],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  6,  24,  41],      │ │
immich_machine_learning  |                              │ │               │   │   [  6,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   [  1,  15,  33],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  14,  32],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      ...,                   │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 40,  67, 101],      │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  55,  89],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  60,  94],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 35,  60,  92],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  58,  90],      │ │
immich_machine_learning  |                              │ │               │   │   [ 43,  68, 100],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 43,  65,  93],      │ │
immich_machine_learning  |                              │ │               │   │   [ 69,  91, 119],      │ │
immich_machine_learning  |                              │ │               │   │   [ 64,  86, 114],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]]],    │ │
immich_machine_learning  |                              │ │               dtype=uint8)                  │ │
immich_machine_learning  |                              │ │   det_scale = 0.3333333333333333            │ │
immich_machine_learning  |                              │ │    im_ratio = 1.3333333333333333            │ │
immich_machine_learning  |                              │ │         img = array([[[  0,  17,  34],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  17,  34],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  18,  35],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  8,   7,   9],      │ │
immich_machine_learning  |                              │ │               │   │   [ 14,  13,  15],      │ │
immich_machine_learning  |                              │ │               │   │   [ 19,  18,  20]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  0,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  2,  20,  37],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  7,   6,   8],      │ │
immich_machine_learning  |                              │ │               │   │   [ 14,  13,  15],      │ │
immich_machine_learning  |                              │ │               │   │   [ 20,  19,  21]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  2,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   [  4,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   [  4,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  5,   4,   6],      │ │
immich_machine_learning  |                              │ │               │   │   [ 14,  13,  15],      │ │
immich_machine_learning  |                              │ │               │   │   [ 22,  21,  23]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      ...,                   │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 32,  56,  86],      │ │
immich_machine_learning  |                              │ │               │   │   [ 36,  60,  90],      │ │
immich_machine_learning  |                              │ │               │   │   [ 42,  66,  96],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  54,  71],      │ │
immich_machine_learning  |                              │ │               │   │   [ 31,  57,  74],      │ │
immich_machine_learning  |                              │ │               │   │   [ 36,  62,  79]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 37,  59,  87],      │ │
immich_machine_learning  |                              │ │               │   │   [ 43,  65,  93],      │ │
immich_microservices     | [Nest] 7  - 03/24/2024, 4:17:44 PM   ERROR [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     | [Nest] 7  - 03/24/2024, 4:17:44 PM   ERROR [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices     |     at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:23:19)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async PersonService.handleDetectFaces (/usr/src/app/dist/domain/person/person.service.js:248:23)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:137:36
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices     | [Nest] 7  - 03/24/2024, 4:17:44 PM   ERROR [JobService] Object:
immich_microservices     | {
immich_microservices     |   "id": "ea7fe00a-068b-46d3-8ae1-61dd6146286f"
immich_microservices     | }
immich_microservices     | 
immich_machine_learning  |                              │ │               │   │   [ 53,  75, 103],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 27,  53,  70],      │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  54,  71],      │ │
immich_machine_learning  |                              │ │               │   │   [ 32,  58,  75]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 44,  66,  94],      │ │
immich_machine_learning  |                              │ │               │   │   [ 50,  72, 100],      │ │
immich_machine_learning  |                              │ │               │   │   [ 61,  83, 111],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 27,  53,  70],      │ │
immich_machine_learning  |                              │ │               │   │   [ 25,  51,  68],      │ │
immich_machine_learning  |                              │ │               │   │   [ 27,  53,  70]]],    │ │
immich_machine_learning  |                              │ │               dtype=uint8)                  │ │
immich_machine_learning  |                              │ │  input_size = (640, 640)                    │ │
immich_machine_learning  |                              │ │     max_num = 0                             │ │
immich_machine_learning  |                              │ │      metric = 'default'                     │ │
immich_machine_learning  |                              │ │ model_ratio = 1.0                           │ │
immich_machine_learning  |                              │ │  new_height = 640                           │ │
immich_machine_learning  |                              │ │   new_width = 480                           │ │
immich_machine_learning  |                              │ │ resized_img = array([[[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  13,  31],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  2,   1,   3],      │ │
immich_machine_learning  |                              │ │               │   │   [  2,   1,   3],      │ │
immich_machine_learning  |                              │ │               │   │   [ 14,  13,  15]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  6,  24,  41],      │ │
immich_machine_learning  |                              │ │               │   │   [  6,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   [  1,  15,  33],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   1],      │ │
immich_machine_learning  |                              │ │               │   │   [  4,   3,   5],      │ │
immich_machine_learning  |                              │ │               │   │   [ 16,  15,  17]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  14,  32],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  2,   1,   3],      │ │
immich_machine_learning  |                              │ │               │   │   [  2,   1,   3],      │ │
immich_machine_learning  |                              │ │               │   │   [ 18,  17,  19]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      ...,                   │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 40,  67, 101],      │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  55,  89],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  60,  94],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 49,  75,  92],      │ │
immich_machine_learning  |                              │ │               │   │   [ 37,  63,  80],      │ │
immich_machine_learning  |                              │ │               │   │   [ 32,  58,  75]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 35,  60,  92],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  58,  90],      │ │
immich_machine_learning  |                              │ │               │   │   [ 43,  68, 100],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 29,  55,  72],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  59,  76],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  59,  76]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 43,  65,  93],      │ │
immich_machine_learning  |                              │ │               │   │   [ 69,  91, 119],      │ │
immich_machine_learning  |                              │ │               │   │   [ 64,  86, 114],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [ 27,  53,  70],      │ │
immich_machine_learning  |                              │ │               │   │   [ 30,  56,  73],      │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  54,  71]]],    │ │
immich_machine_learning  |                              │ │               dtype=uint8)                  │ │
immich_machine_learning  |                              │ │        self = <insightface.model_zoo.retin… │ │
immich_machine_learning  |                              │ │               object at 0x7ab23c362dd0>     │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:152 in forward      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   149 │   │   kpss_list = []                    │
immich_machine_learning  |                              │   150 │   │   input_size = tuple(img.shape[0:2] │
immich_machine_learning  |                              │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
immich_machine_learning  |                              │       (self.input_mean, self.input_mean, self.i │
immich_machine_learning  |                              │ ❱ 152 │   │   net_outs = self.session.run(self. │
immich_machine_learning  |                              │   153 │   │                                     │
immich_machine_learning  |                              │   154 │   │   input_height = blob.shape[2]      │
immich_machine_learning  |                              │   155 │   │   input_width = blob.shape[3]       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │ bboxes_list = []                            │ │
immich_machine_learning  |                              │ │        blob = array([[[[-0.71484375,        │ │
immich_machine_learning  |                              │ │               -0.71484375, -0.75390625,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.67578125,        │ │
immich_machine_learning  |                              │ │               -0.69140625, -0.73828125,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.71484375,        │ │
immich_machine_learning  |                              │ │               -0.71484375, -0.74609375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    ...,                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.20703125,        │ │
immich_machine_learning  |                              │ │               -0.30078125, -0.26171875,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.27734375,        │ │
immich_machine_learning  |                              │ │               -0.29296875, -0.21484375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.26953125,        │ │
immich_machine_learning  |                              │ │               -0.06640625, -0.10546875,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375]],                │ │
immich_machine_learning  |                              │ │               │   │                         │ │
immich_machine_learning  |                              │ │               │   │   [[-0.84765625,        │ │
immich_machine_learning  |                              │ │               -0.84765625, -0.89453125,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.80859375,        │ │
immich_machine_learning  |                              │ │               -0.82421875, -0.87890625,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.84765625,        │ │
immich_machine_learning  |                              │ │               -0.84765625, -0.88671875,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    ...,                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.47265625,        │ │
immich_machine_learning  |                              │ │               -0.56640625, -0.52734375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.52734375,        │ │
immich_machine_learning  |                              │ │               -0.54296875, -0.46484375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.48828125,        │ │
immich_machine_learning  |                              │ │               -0.28515625, -0.32421875,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375]],                │ │
immich_machine_learning  |                              │ │               │   │                         │ │
immich_machine_learning  |                              │ │               │   │   [[-0.98828125,        │ │
immich_machine_learning  |                              │ │               -0.97265625, -0.99609375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.94921875,        │ │
immich_machine_learning  |                              │ │               -0.94921875, -0.98828125,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.98828125,        │ │
immich_machine_learning  |                              │ │               -0.97265625, -0.99609375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    ...,                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.68359375,        │ │
immich_machine_learning  |                              │ │               -0.77734375, -0.73828125,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.72265625,        │ │
immich_machine_learning  |                              │ │               -0.73828125, -0.66015625,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375],                 │ │
immich_machine_learning  |                              │ │               │   │    [-0.66015625,        │ │
immich_machine_learning  |                              │ │               -0.45703125, -0.49609375,     │ │
immich_machine_learning  |                              │ │               ..., -0.99609375,             │ │
immich_machine_learning  |                              │ │               │   │     -0.99609375,        │ │
immich_machine_learning  |                              │ │               -0.99609375]]]],              │ │
immich_machine_learning  |                              │ │               dtype=float32)                │ │
immich_machine_learning  |                              │ │         img = array([[[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  13,  31],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  6,  24,  41],      │ │
immich_machine_learning  |                              │ │               │   │   [  6,  22,  39],      │ │
immich_machine_learning  |                              │ │               │   │   [  1,  15,  33],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[  1,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  3,  19,  36],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,  14,  32],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      ...,                   │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 40,  67, 101],      │ │
immich_machine_learning  |                              │ │               │   │   [ 28,  55,  89],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  60,  94],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 35,  60,  92],      │ │
immich_machine_learning  |                              │ │               │   │   [ 33,  58,  90],      │ │
immich_machine_learning  |                              │ │               │   │   [ 43,  68, 100],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]],     │ │
immich_machine_learning  |                              │ │               │                             │ │
immich_machine_learning  |                              │ │               │      [[ 43,  65,  93],      │ │
immich_machine_learning  |                              │ │               │   │   [ 69,  91, 119],      │ │
immich_machine_learning  |                              │ │               │   │   [ 64,  86, 114],      │ │
immich_machine_learning  |                              │ │               │   │   ...,                  │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0],      │ │
immich_machine_learning  |                              │ │               │   │   [  0,   0,   0]]],    │ │
immich_machine_learning  |                              │ │               dtype=uint8)                  │ │
immich_machine_learning  |                              │ │  input_size = (640, 640)                    │ │
immich_machine_learning  |                              │ │   kpss_list = []                            │ │
immich_machine_learning  |                              │ │ scores_list = []                            │ │
immich_machine_learning  |                              │ │        self = <insightface.model_zoo.retin… │ │
immich_machine_learning  |                              │ │               object at 0x7ab23c362dd0>     │ │
immich_machine_learning  |                              │ │   threshold = 0.7                           │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:22 │
immich_machine_learning  |                              │ 0 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    217 │   │   if not output_names:             │
immich_machine_learning  |                              │    218 │   │   │   output_names = [output.name  │
immich_machine_learning  |                              │    219 │   │   try:                             │
immich_machine_learning  |                              │ ❱  220 │   │   │   return self._sess.run(output │
immich_machine_learning  |                              │    221 │   │   except C.EPFail as err:          │
immich_machine_learning  |                              │    222 │   │   │   if self._enable_fallback:    │
immich_machine_learning  |                              │    223 │   │   │   │   print(f"EP Error: {err!s │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ ╭────────────────── locals ───────────────────╮ │
immich_machine_learning  |                              │ │   input_feed = {                            │ │
immich_machine_learning  |                              │ │                │   'input.1':               │ │
immich_machine_learning  |                              │ │                array([[[[-0.71484375,       │ │
immich_machine_learning  |                              │ │                -0.71484375, -0.75390625,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.67578125,       │ │
immich_machine_learning  |                              │ │                -0.69140625, -0.73828125,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.71484375,       │ │
immich_machine_learning  |                              │ │                -0.71484375, -0.74609375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    ...,                │ │
immich_machine_learning  |                              │ │                │   │    [-0.20703125,       │ │
immich_machine_learning  |                              │ │                -0.30078125, -0.26171875,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.27734375,       │ │
immich_machine_learning  |                              │ │                -0.29296875, -0.21484375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.26953125,       │ │
immich_machine_learning  |                              │ │                -0.06640625, -0.10546875,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375]],               │ │
immich_machine_learning  |                              │ │                │   │                        │ │
immich_machine_learning  |                              │ │                │   │   [[-0.84765625,       │ │
immich_machine_learning  |                              │ │                -0.84765625, -0.89453125,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.80859375,       │ │
immich_machine_learning  |                              │ │                -0.82421875, -0.87890625,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.84765625,       │ │
immich_machine_learning  |                              │ │                -0.84765625, -0.88671875,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    ...,                │ │
immich_machine_learning  |                              │ │                │   │    [-0.47265625,       │ │
immich_machine_learning  |                              │ │                -0.56640625, -0.52734375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.52734375,       │ │
immich_machine_learning  |                              │ │                -0.54296875, -0.46484375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.48828125,       │ │
immich_machine_learning  |                              │ │                -0.28515625, -0.32421875,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375]],               │ │
immich_machine_learning  |                              │ │                │   │                        │ │
immich_machine_learning  |                              │ │                │   │   [[-0.98828125,       │ │
immich_machine_learning  |                              │ │                -0.97265625, -0.99609375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.94921875,       │ │
immich_machine_learning  |                              │ │                -0.94921875, -0.98828125,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.98828125,       │ │
immich_machine_learning  |                              │ │                -0.97265625, -0.99609375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    ...,                │ │
immich_machine_learning  |                              │ │                │   │    [-0.68359375,       │ │
immich_machine_learning  |                              │ │                -0.77734375, -0.73828125,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.72265625,       │ │
immich_machine_learning  |                              │ │                -0.73828125, -0.66015625,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375],                │ │
immich_machine_learning  |                              │ │                │   │    [-0.66015625,       │ │
immich_machine_learning  |                              │ │                -0.45703125, -0.49609375,    │ │
immich_machine_learning  |                              │ │                ..., -0.99609375,            │ │
immich_machine_learning  |                              │ │                │   │     -0.99609375,       │ │
immich_machine_learning  |                              │ │                -0.99609375]]]],             │ │
immich_machine_learning  |                              │ │                dtype=float32)               │ │
immich_machine_learning  |                              │ │                }                            │ │
immich_machine_learning  |                              │ │ output_names = [                            │ │
immich_machine_learning  |                              │ │                │   '448',                   │ │
immich_machine_learning  |                              │ │                │   '471',                   │ │
immich_machine_learning  |                              │ │                │   '494',                   │ │
immich_machine_learning  |                              │ │                │   '451',                   │ │
immich_machine_learning  |                              │ │                │   '474',                   │ │
immich_machine_learning  |                              │ │                │   '497',                   │ │
immich_machine_learning  |                              │ │                │   '454',                   │ │
immich_machine_learning  |                              │ │                │   '477',                   │ │
immich_machine_learning  |                              │ │                │   '500'                    │ │
immich_machine_learning  |                              │ │                ]                            │ │
immich_machine_learning  |                              │ │  run_options = None                         │ │
immich_machine_learning  |                              │ │         self = <onnxruntime.capi.onnxrunti… │ │
immich_machine_learning  |                              │ │                object at 0x7ab23c362b00>    │ │
immich_machine_learning  |                              │ ╰─────────────────────────────────────────────╯ │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              RuntimeException: [ONNXRuntimeError] : 6 :         
immich_machine_learning  |                              RUNTIME_EXCEPTION : Encountered unknown exception  
immich_machine_learning  |                              in Run()                                           
immich_machine_learning  | [03/24/24 16:17:48] DEBUG    Checking for inactivity...                         
mertalev commented 3 months ago

I made an upstream issue for this based on the info here. Let's wait for their response.

yu-jingrui commented 3 months ago

I am having this problem too.

diogovalentte commented 3 months ago

I'm having the same issue in 1.100.0 too.

mertalev commented 3 months ago

For future reference, it's better to leave a thumbs up for issues if they affect you. Leaving a comment notifies every participant in the issue, so it should be reserved for comments that drive the issue forward.

kaienli commented 3 months ago

same issue in 1.100.0 too. on wsl2,my config is immich-machine-learning: container_name: immich_machine_learning restart: always image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino devices:

agrawalsourav98 commented 3 months ago

I am also facing this issue. It runs fine on my laptop which has i5-1145G7 but doesn't work on my immich desktop which has i3-12100.

Can we provide an option to disable OpenVINO runtime for where it fails only for the facial recognition model? Because the smart search models work fine.

mertalev commented 2 months ago

The i5-1145G7 has Iris graphics, same with the CPU I tested with. It seems to be that UHD graphics doesn't work, but Iris does.

jsapede commented 2 months ago

same here with 1.101.0 on i5 Gen 7 was perfectly working on older releases

plmsuper8 commented 2 months ago

similar here with smart search. 1.102.3 on N5095. sadly the smart search nerver works for me.

good news, when change from immich-machine-learning:${IMMICH_VERSION}-openvino to immich-machine-learning:${IMMICH_VERSION}, it works! (related https://github.com/immich-app/immich/issues/8918)

jsapede commented 2 months ago

same error on 1.102.3, was working until 1.099

immich_machine_learning | [04/29/24 12:59:22] INFO Setting 'buffalo_l' execution providers to immich_machine_learning | ['OpenVINOExecutionProvider', immich_machine_learning | 'CPUExecutionProvider'], in descending order of immich_machine_learning | preference immich_machine_learning | [04/29/24 12:59:22] INFO Loading facial recognition model 'buffalo_l' to immich_machine_learning | memory immich_machine_learning | [04/29/24 12:59:29] ERROR Exception in ASGI application immich_machine_learning | immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:118 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 115 │ │ immich_machine_learning | │ 116 │ model = await load(await model_cache. │ immich_machine_learning | │ ttl=settings.model_ttl, **kwargs)) │ immich_machine_learning | │ 117 │ model.configure(**kwargs) │ immich_machine_learning | │ ❱ 118 │ outputs = await run(model.predict, in │ immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │ immich_machine_learning | │ 120 │ immich_machine_learning | │ 121 │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:125 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │ immich_machine_learning | │ 123 │ if thread_pool is None: │ immich_machine_learning | │ 124 │ │ return func(inputs) │ immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │ immich_machine_learning | │ 126 │ immich_machine_learning | │ 127 │ immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:59 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 56 │ │ self.load() │ immich_machine_learning | │ 57 │ │ if model_kwargs: │ immich_machine_learning | │ 58 │ │ │ self.configure(**model_kwargs │ immich_machine_learning | │ ❱ 59 │ │ return self._predict(inputs) │ immich_machine_learning | │ 60 │ │ immich_machine_learning | │ 61 │ @abstractmethod │ immich_machine_learning | │ 62 │ def _predict(self, inputs: Any) -> An │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/facial_recognition.py:49 in │ immich_machine_learning | │ _predict │ immich_machine_learning | │ │ immich_machine_learning | │ 46 │ │ else: │ immich_machine_learning | │ 47 │ │ │ decoded_image = image │ immich_machine_learning | │ 48 │ │ assert is_ndarray(decoded_image, n │ immich_machine_learning | │ ❱ 49 │ │ bboxes, kpss = self.det_model.dete │ immich_machine_learning | │ 50 │ │ if bboxes.size == 0: │ immich_machine_learning | │ 51 │ │ │ return [] │ immich_machine_learning | │ 52 │ │ assert is_ndarray(kpss, np.float32 │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/insightf │ immich_machine_learning | │ ace/model_zoo/retinaface.py:224 in detect │ immich_machine_learning | │ │ immich_machine_learning | │ 221 │ │ det_img = np.zeros( (input_size[1 │ immich_machine_learning | │ 222 │ │ det_img[:new_height, :new_width, │ immich_machine_learning | │ 223 │ │ │ immich_machine_learning | │ ❱ 224 │ │ scores_list, bboxes_list, kpss_li │ immich_machine_learning | │ 225 │ │ │ immich_machine_learning | │ 226 │ │ scores = np.vstack(scores_list) │ immich_machine_learning | │ 227 │ │ scores_ravel = scores.ravel() │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/insightf │ immich_machine_learning | │ ace/model_zoo/retinaface.py:152 in forward │ immich_machine_learning | │ │ immich_machine_learning | │ 149 │ │ kpss_list = [] │ immich_machine_learning | │ 150 │ │ input_size = tuple(img.shape[0:2] │ immich_machine_learning | │ 151 │ │ blob = cv2.dnn.blobFromImage(img, │ immich_machine_learning | │ (self.input_mean, self.input_mean, self.i │ immich_machine_learning | │ ❱ 152 │ │ net_outs = self.session.run(self. │ immich_machine_learning | │ 153 │ │ │ immich_machine_learning | │ 154 │ │ input_height = blob.shape[2] │ immich_machine_learning | │ 155 │ │ input_width = blob.shape[3] │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:22 │ immich_machine_learning | │ 0 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 217 │ │ if not output_names: │ immich_machine_learning | │ 218 │ │ │ output_names = [output.name │ immich_machine_learning | │ 219 │ │ try: │ immich_machine_learning | │ ❱ 220 │ │ │ return self._sess.run(output │ immich_machine_learning | │ 221 │ │ except C.EPFail as err: │ immich_machine_learning | │ 222 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 223 │ │ │ │ print(f"EP Error: {err!s │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | RuntimeException: [ONNXRuntimeError] : 6 : immich_machine_learning | RUNTIME_EXCEPTION : Encountered unknown exception immich_machine_learning | in Run() immich_microservices | [Nest] 7 - 04/29/2024, 12:59:29 PM ERROR [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error immich_microservices | [Nest] 7 - 04/29/2024, 12:59:29 PM ERROR [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:268:23) immich_microservices | at async /usr/src/app/dist/services/job.service.js:149:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 7 - 04/29/2024, 12:59:29 PM ERROR [JobService] Object: immich_microservices | { immich_microservices | "id": "541c6706-c6bd-43e0-bf74-d8bbc4d1b664" immich_microservices | } immich_microservices |

nodis commented 2 months ago

me too

engels0n commented 2 months ago

Same problem in new version 1.103.1

`[05/02/24 07:05:03] INFO Setting 'antelopev2' execution providers to ['OpenVINOExecutionProvider', 'CPUExecutionProvider'], in descending order of preference [05/02/24 07:05:03] INFO Loading facial recognition model 'antelopev2' to memory [05/02/24 07:05:11] ERROR Exception in ASGI application

                         ╭─────── Traceback (most recent call last) ───────╮
                         │ /usr/src/app/main.py:118 in predict             │
                         │                                                 │
                         │   115 │                                         │
                         │   116 │   model = await load(await model_cache. │
                         │       ttl=settings.model_ttl, **kwargs))        │
                         │   117 │   model.configure(**kwargs)             │
                         │ ❱ 118 │   outputs = await run(model.predict, in │
                         │   119 │   return ORJSONResponse(outputs)        │
                         │   120                                           │
                         │   121                                           │
                         │                                                 │
                         │ /usr/src/app/main.py:125 in run                 │
                         │                                                 │
                         │   122 async def run(func: Callable[..., Any], i │
                         │   123 │   if thread_pool is None:               │
                         │   124 │   │   return func(inputs)               │
                         │ ❱ 125 │   return await asyncio.get_running_loop │
                         │   126                                           │
                         │   127                                           │
                         │   128 async def load(model: InferenceModel) ->  │
                         │                                                 │
                         │ /usr/lib/python3.10/concurrent/futures/thread.p │
                         │ y:58 in run                                     │
                         │                                                 │
                         │ /usr/src/app/models/base.py:59 in predict       │
                         │                                                 │
                         │    56 │   │   self.load()                       │
                         │    57 │   │   if model_kwargs:                  │
                         │    58 │   │   │   self.configure(**model_kwargs │
                         │ ❱  59 │   │   return self._predict(inputs)      │
                         │    60 │                                         │
                         │    61 │   @abstractmethod                       │
                         │    62 │   def _predict(self, inputs: Any) -> An │
                         │                                                 │
                         │ /usr/src/app/models/facial_recognition.py:49 in │
                         │ _predict                                        │
                         │                                                 │
                         │   46 │   │   else:                              │
                         │   47 │   │   │   decoded_image = image          │
                         │   48 │   │   assert is_ndarray(decoded_image, n │
                         │ ❱ 49 │   │   bboxes, kpss = self.det_model.dete │
                         │   50 │   │   if bboxes.size == 0:               │
                         │   51 │   │   │   return []                      │
                         │   52 │   │   assert is_ndarray(kpss, np.float32 │
                         │                                                 │
                         │ /opt/venv/lib/python3.10/site-packages/insightf │
                         │ ace/model_zoo/retinaface.py:224 in detect       │
                         │                                                 │
                         │   221 │   │   det_img = np.zeros( (input_size[1 │
                         │   222 │   │   det_img[:new_height, :new_width,  │
                         │   223 │   │                                     │
                         │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
                         │   225 │   │                                     │
                         │   226 │   │   scores = np.vstack(scores_list)   │
                         │   227 │   │   scores_ravel = scores.ravel()     │
                         │                                                 │
                         │ /opt/venv/lib/python3.10/site-packages/insightf │
                         │ ace/model_zoo/retinaface.py:152 in forward      │
                         │                                                 │
                         │   149 │   │   kpss_list = []                    │
                         │   150 │   │   input_size = tuple(img.shape[0:2] │
                         │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
                         │       (self.input_mean, self.input_mean, self.i │
                         │ ❱ 152 │   │   net_outs = self.session.run(self. │
                         │   153 │   │                                     │
                         │   154 │   │   input_height = blob.shape[2]      │
                         │   155 │   │   input_width = blob.shape[3]       │
                         │                                                 │
                         │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
                         │ ime/capi/onnxruntime_inference_collection.py:22 │
                         │ 0 in run                                        │
                         │                                                 │
                         │    217 │   │   if not output_names:             │
                         │    218 │   │   │   output_names = [output.name  │
                         │    219 │   │   try:                             │
                         │ ❱  220 │   │   │   return self._sess.run(output │
                         │    221 │   │   except C.EPFail as err:          │
                         │    222 │   │   │   if self._enable_fallback:    │
                         │    223 │   │   │   │   print(f"EP Error: {err!s │
                         ╰─────────────────────────────────────────────────╯
                         RuntimeException: [ONNXRuntimeError] : 6 :
                         RUNTIME_EXCEPTION : Encountered unknown exception
                         in Run()`
engels0n commented 2 months ago

@plmsuper8

Thanks for the hint and "workaround". But now it obviously uses the CPU instead of the GPU ;) - so the problem still exists.

jsapede commented 1 month ago

still not working in 1.104.0 will there be a solution for openvino use ?

Snuupy commented 1 month ago

I was going to try downgrading onnxruntime to 2024.1 (from the current 2024.3) in the immich-machine-learning Dockerfile. The last version to have working openvino was .98 which used 2024.1. I was going to do a build and test it, but I haven't had time yet.

Edit: nope, not that simple if you try to edit the Dockerfile and change just the onnxruntime version unfortunately.

logs ``` immich_microservices | [Nest] 6 - 05/14/2024, 6:24:58 PM LOG [ImmichMicroservices] [MediaService] Successfully generated WEBP image thumbnail for asset b5bd0dc0-d6c7-4133-8a6c-59fd9dfe7e4c immich_machine_learning | [05/14/24 18:24:58] INFO Setting 'ViT-B-32__openai' execution providers to immich_machine_learning | ['OpenVINOExecutionProvider', immich_machine_learning | 'CPUExecutionProvider'], in descending order of immich_machine_learning | preference immich_machine_learning | [05/14/24 18:24:58] INFO Setting 'buffalo_l' execution providers to immich_machine_learning | ['OpenVINOExecutionProvider', immich_machine_learning | 'CPUExecutionProvider'], in descending order of immich_machine_learning | preference immich_machine_learning | [05/14/24 18:24:58] INFO Loading clip model 'ViT-B-32__openai' to memory immich_machine_learning | 2024-05-14 18:24:59.955630642 [E:onnxruntime:, inference_session.cc:1985 Initialize] Encountered unknown exception in Initialize() immich_machine_learning | [05/14/24 18:24:59] ERROR Exception in ASGI application immich_machine_learning | immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:116 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 113 │ except orjson.JSONDecodeError: │ immich_machine_learning | │ 114 │ │ raise HTTPException(400, f"Invali │ immich_machine_learning | │ 115 │ │ immich_machine_learning | │ ❱ 116 │ model = await load(await model_cache. │ immich_machine_learning | │ ttl=settings.model_ttl, **kwargs)) │ immich_machine_learning | │ 117 │ model.configure(**kwargs) │ immich_machine_learning | │ 118 │ outputs = await run(model.predict, in │ immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:137 in load │ immich_machine_learning | │ │ immich_machine_learning | │ 134 │ │ │ model.load() │ immich_machine_learning | │ 135 │ │ immich_machine_learning | │ 136 │ try: │ immich_machine_learning | │ ❱ 137 │ │ await run(_load, model) │ immich_machine_learning | │ 138 │ │ return model │ immich_machine_learning | │ 139 │ except (OSError, InvalidProtobuf, Bad │ immich_machine_learning | │ 140 │ │ log.warning( │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:125 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │ immich_machine_learning | │ 123 │ if thread_pool is None: │ immich_machine_learning | │ 124 │ │ return func(inputs) │ immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │ immich_machine_learning | │ 126 │ immich_machine_learning | │ 127 │ immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:134 in _load │ immich_machine_learning | │ │ immich_machine_learning | │ 131 │ │ immich_machine_learning | │ 132 │ def _load(model: InferenceModel) -> N │ immich_machine_learning | │ 133 │ │ with lock: │ immich_machine_learning | │ ❱ 134 │ │ │ model.load() │ immich_machine_learning | │ 135 │ │ immich_machine_learning | │ 136 │ try: │ immich_machine_learning | │ 137 │ │ await run(_load, model) │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:52 in load │ immich_machine_learning | │ │ immich_machine_learning | │ 49 │ │ │ return │ immich_machine_learning | │ 50 │ │ self.download() │ immich_machine_learning | │ 51 │ │ log.info(f"Loading {self.model_ty │ immich_machine_learning | │ to memory") │ immich_machine_learning | │ ❱ 52 │ │ self._load() │ immich_machine_learning | │ 53 │ │ self.loaded = True │ immich_machine_learning | │ 54 │ │ immich_machine_learning | │ 55 │ def predict(self, inputs: Any, **mode │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/clip.py:146 in _load │ immich_machine_learning | │ │ immich_machine_learning | │ 143 │ │ super().__init__(clean_name(model │ immich_machine_learning | │ 144 │ │ immich_machine_learning | │ 145 │ def _load(self) -> None: │ immich_machine_learning | │ ❱ 146 │ │ super()._load() │ immich_machine_learning | │ 147 │ │ self._load_tokenizer() │ immich_machine_learning | │ 148 │ │ │ immich_machine_learning | │ 149 │ │ size: list[int] | int = self.prep │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/clip.py:41 in _load │ immich_machine_learning | │ │ immich_machine_learning | │ 38 │ │ │ immich_machine_learning | │ 39 │ │ if self.mode == "vision" or self. │ immich_machine_learning | │ 40 │ │ │ log.debug(f"Loading clip visi │ immich_machine_learning | │ ❱ 41 │ │ │ self.vision_model = self._mak │ immich_machine_learning | │ 42 │ │ │ log.debug(f"Loaded clip visio │ immich_machine_learning | │ 43 │ │ immich_machine_learning | │ 44 │ def _predict(self, image_or_text: Ima │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:117 in │ immich_machine_learning | │ _make_session │ immich_machine_learning | │ │ immich_machine_learning | │ 114 │ │ │ case ".armnn": │ immich_machine_learning | │ 115 │ │ │ │ session = AnnSession(mode │ immich_machine_learning | │ 116 │ │ │ case ".onnx": │ immich_machine_learning | │ ❱ 117 │ │ │ │ session = ort.InferenceSe │ immich_machine_learning | │ 118 │ │ │ │ │ model_path.as_posix() │ immich_machine_learning | │ 119 │ │ │ │ │ sess_options=self.ses │ immich_machine_learning | │ 120 │ │ │ │ │ providers=self.provid │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:41 │ immich_machine_learning | │ 9 in __init__ │ immich_machine_learning | │ │ immich_machine_learning | │ 416 │ │ disabled_optimizers = kwargs["di │ immich_machine_learning | │ kwargs else None │ immich_machine_learning | │ 417 │ │ │ immich_machine_learning | │ 418 │ │ try: │ immich_machine_learning | │ ❱ 419 │ │ │ self._create_inference_sessi │ immich_machine_learning | │ disabled_optimizers) │ immich_machine_learning | │ 420 │ │ except (ValueError, RuntimeError │ immich_machine_learning | │ 421 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 422 │ │ │ │ try: │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:48 │ immich_machine_learning | │ 3 in _create_inference_session │ immich_machine_learning | │ │ immich_machine_learning | │ 480 │ │ │ disabled_optimizers = set(di │ immich_machine_learning | │ 481 │ │ │ immich_machine_learning | │ 482 │ │ # initialize the C++ InferenceSe │ immich_machine_learning | │ ❱ 483 │ │ sess.initialize_session(provider │ immich_machine_learning | │ 484 │ │ │ immich_machine_learning | │ 485 │ │ self._sess = sess │ immich_machine_learning | │ 486 │ │ self._sess_options = self._sess. │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | RuntimeException: [ONNXRuntimeError] : 6 : immich_machine_learning | RUNTIME_EXCEPTION : Encountered unknown exception immich_machine_learning | in Initialize() immich_machine_learning | [05/14/24 18:24:59] INFO Loading facial recognition model 'buffalo_l' to immich_machine_learning | memory immich_microservices | [Nest] 6 - 05/14/2024, 6:25:00 PM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | [Nest] 6 - 05/14/2024, 6:25:00 PM ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:85:31) immich_microservices | at async /usr/src/app/dist/services/job.service.js:149:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 6 - 05/14/2024, 6:25:00 PM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "b5bd0dc0-d6c7-4133-8a6c-59fd9dfe7e4c", immich_microservices | "source": "upload" immich_microservices | } immich_microservices | immich_machine_learning | 2024-05-14 18:25:00.970047176 [E:onnxruntime:, inference_session.cc:1985 Initialize] Encountered unknown exception in Initialize() immich_machine_learning | [05/14/24 18:25:00] ERROR Exception in ASGI application immich_machine_learning | immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:116 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 113 │ except orjson.JSONDecodeError: │ immich_machine_learning | │ 114 │ │ raise HTTPException(400, f"Invali │ immich_machine_learning | │ 115 │ │ immich_machine_learning | │ ❱ 116 │ model = await load(await model_cache. │ immich_machine_learning | │ ttl=settings.model_ttl, **kwargs)) │ immich_machine_learning | │ 117 │ model.configure(**kwargs) │ immich_machine_learning | │ 118 │ outputs = await run(model.predict, in │ immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:137 in load │ immich_machine_learning | │ │ immich_machine_learning | │ 134 │ │ │ model.load() │ immich_machine_learning | │ 135 │ │ immich_machine_learning | │ 136 │ try: │ immich_machine_learning | │ ❱ 137 │ │ await run(_load, model) │ immich_machine_learning | │ 138 │ │ return model │ immich_machine_learning | │ 139 │ except (OSError, InvalidProtobuf, Bad │ immich_machine_learning | │ 140 │ │ log.warning( │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:125 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │ immich_machine_learning | │ 123 │ if thread_pool is None: │ immich_machine_learning | │ 124 │ │ return func(inputs) │ immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │ immich_machine_learning | │ 126 │ immich_machine_learning | │ 127 │ immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:134 in _load │ immich_machine_learning | │ │ immich_machine_learning | │ 131 │ │ immich_machine_learning | │ 132 │ def _load(model: InferenceModel) -> N │ immich_machine_learning | │ 133 │ │ with lock: │ immich_machine_learning | │ ❱ 134 │ │ │ model.load() │ immich_machine_learning | │ 135 │ │ immich_machine_learning | │ 136 │ try: │ immich_machine_learning | │ 137 │ │ await run(_load, model) │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:52 in load │ immich_machine_learning | │ │ immich_machine_learning | │ 49 │ │ │ return │ immich_machine_learning | │ 50 │ │ self.download() │ immich_machine_learning | │ 51 │ │ log.info(f"Loading {self.model_ty │ immich_machine_learning | │ to memory") │ immich_machine_learning | │ ❱ 52 │ │ self._load() │ immich_machine_learning | │ 53 │ │ self.loaded = True │ immich_machine_learning | │ 54 │ │ immich_machine_learning | │ 55 │ def predict(self, inputs: Any, **mode │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/facial_recognition.py:30 in │ immich_machine_learning | │ _load │ immich_machine_learning | │ │ immich_machine_learning | │ 27 │ │ super().__init__(clean_name(model_ │ immich_machine_learning | │ 28 │ │ immich_machine_learning | │ 29 │ def _load(self) -> None: │ immich_machine_learning | │ ❱ 30 │ │ self.det_model = RetinaFace(sessio │ immich_machine_learning | │ 31 │ │ self.rec_model = ArcFaceONNX( │ immich_machine_learning | │ 32 │ │ │ self.rec_file.with_suffix(".on │ immich_machine_learning | │ 33 │ │ │ session=self._make_session(sel │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:117 in │ immich_machine_learning | │ _make_session │ immich_machine_learning | │ │ immich_machine_learning | │ 114 │ │ │ case ".armnn": │ immich_machine_learning | │ 115 │ │ │ │ session = AnnSession(mode │ immich_machine_learning | │ 116 │ │ │ case ".onnx": │ immich_machine_learning | │ ❱ 117 │ │ │ │ session = ort.InferenceSe │ immich_machine_learning | │ 118 │ │ │ │ │ model_path.as_posix() │ immich_machine_learning | │ 119 │ │ │ │ │ sess_options=self.ses │ immich_machine_learning | │ 120 │ │ │ │ │ providers=self.provid │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:41 │ immich_machine_learning | │ 9 in __init__ │ immich_machine_learning | │ │ immich_machine_learning | │ 416 │ │ disabled_optimizers = kwargs["di │ immich_machine_learning | │ kwargs else None │ immich_machine_learning | │ 417 │ │ │ immich_machine_learning | │ 418 │ │ try: │ immich_machine_learning | │ ❱ 419 │ │ │ self._create_inference_sessi │ immich_machine_learning | │ disabled_optimizers) │ immich_machine_learning | │ 420 │ │ except (ValueError, RuntimeError │ immich_machine_learning | │ 421 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 422 │ │ │ │ try: │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:48 │ immich_machine_learning | │ 3 in _create_inference_session │ immich_machine_learning | │ │ immich_machine_learning | │ 480 │ │ │ disabled_optimizers = set(di │ immich_machine_learning | │ 481 │ │ │ immich_machine_learning | │ 482 │ │ # initialize the C++ InferenceSe │ immich_machine_learning | │ ❱ 483 │ │ sess.initialize_session(provider │ immich_machine_learning | │ 484 │ │ │ immich_machine_learning | │ 485 │ │ self._sess = sess │ immich_machine_learning | │ 486 │ │ self._sess_options = self._sess. │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | RuntimeException: [ONNXRuntimeError] : 6 : immich_machine_learning | RUNTIME_EXCEPTION : Encountered unknown exception immich_machine_learning | in Initialize() immich_microservices | [Nest] 6 - 05/14/2024, 6:25:01 PM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error immich_microservices | [Nest] 6 - 05/14/2024, 6:25:01 PM ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:269:23) immich_microservices | at async /usr/src/app/dist/services/job.service.js:149:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 6 - 05/14/2024, 6:25:01 PM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "b5bd0dc0-d6c7-4133-8a6c-59fd9dfe7e4c", immich_microservices | "source": "upload" immich_microservices | } immich_microservices | ```
Snuupy commented 1 month ago

I got it working, give me a bit to upload the image and do a writeup :) Thanks @mertalev for the help!

Snuupy commented 1 month ago

Want to use my image?

In your docker-compose.yml, replace

image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino with image: ghcr.io/snuupy/immich-machine-learning:v1.105.0-openvino

https://github.com/Snuupy/immich/pkgs/container/immich-machine-learning

Want to build your own image? Here's how I did it:

install poetry, change onnxruntime-openvino to target 1.15 or 1.16 in the pyproject.toml file, then run poetry lock --no-update

[tool.poetry.group.openvino.dependencies]
# onnxruntime-openvino = "^1.17.1"
onnxruntime-openvino = ">=1.15.0,<1.16.0"

In the machine-learning/Dockerfile, change the line:

FROM openvino/ubuntu22_runtime:2023.3.0@sha256:176646df619032ea6c10faf842867119c393e7497b7f88b5e307e932a0fd5aa8 as builder-openvino (v1.105.0)

to FROM openvino/ubuntu22_runtime:2023.1.0@sha256:002842a9005ba01543b7169ff6f14ecbec82287f09c4d1dd37717f0a8e8754a7 as builder-openvino (v1.98.2)

then do a docker build --build-arg="DEVICE=openvino" -t NAMESPACE/immich-machine-learning:v1.105.0-openvino .

jsapede commented 1 month ago

tested this morning seems to work, its scrapping !

engels0n commented 1 month ago

Nice ! Thank you @Snuupy

@alextran1502 can you merge it into the next version, please? ;)

Snuupy commented 1 month ago

Nice ! Thank you @Snuupy

@alextran1502 can you merge it into the next version, please? ;)

No because it will break openvino for other users (XE/ARC) afaik

Snuupy commented 1 month ago

I will build 105.1 soon™️

jsapede commented 1 month ago

hello, using snuppy trick did the job for facial recognition however it still stands errors on smart search :

immich_machine_learning | 2024-05-15 07:52:52.642944782 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running OpenVINO-EP-subgraph_2 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0' Status Message: /home/onnxruntimedev/onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:53 onnxruntime::openvino_ep::OVExeNetwork onnxruntime::openvino_ep::OVCore::LoadNetwork(const string&, std::string&, ov::AnyMap&, std::string) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0Check 'false' failed at src/inference/src/core.cpp:149: immich_machine_learning | invalid external data: ExternalDataInfo(data_full_path: Constant_2611_attr__value, offset: 0, data_length: 0) immich_machine_learning | immich_machine_learning | immich_machine_learning | [05/15/24 07:52:52] ERROR Exception in ASGI application immich_machine_learning | immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:118 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 115 │ │ immich_machine_learning | │ 116 │ model = await load(await model_cache. │ immich_machine_learning | │ ttl=settings.model_ttl, **kwargs)) │ immich_machine_learning | │ 117 │ model.configure(**kwargs) │ immich_machine_learning | │ ❱ 118 │ outputs = await run(model.predict, in │ immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │ immich_machine_learning | │ 120 │ immich_machine_learning | │ 121 │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:125 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │ immich_machine_learning | │ 123 │ if thread_pool is None: │ immich_machine_learning | │ 124 │ │ return func(inputs) │ immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │ immich_machine_learning | │ 126 │ immich_machine_learning | │ 127 │ immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:59 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 56 │ │ self.load() │ immich_machine_learning | │ 57 │ │ if model_kwargs: │ immich_machine_learning | │ 58 │ │ │ self.configure(**model_kwargs │ immich_machine_learning | │ ❱ 59 │ │ return self._predict(inputs) │ immich_machine_learning | │ 60 │ │ immich_machine_learning | │ 61 │ @abstractmethod │ immich_machine_learning | │ 62 │ def _predict(self, inputs: Any) -> An │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/clip.py:52 in _predict │ immich_machine_learning | │ │ immich_machine_learning | │ 49 │ │ │ case Image.Image(): │ immich_machine_learning | │ 50 │ │ │ │ if self.mode == "text": │ immich_machine_learning | │ 51 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ ❱ 52 │ │ │ │ outputs: NDArray[np.float │ immich_machine_learning | │ self.transform(image_or_text))[0][0] │ immich_machine_learning | │ 53 │ │ │ case str(): │ immich_machine_learning | │ 54 │ │ │ │ if self.mode == "vision": │ immich_machine_learning | │ 55 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:21 │ immich_machine_learning | │ 9 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 216 │ │ if not output_names: │ immich_machine_learning | │ 217 │ │ │ output_names = [output.name f │ immich_machine_learning | │ 218 │ │ try: │ immich_machine_learning | │ ❱ 219 │ │ │ return self._sess.run(output_ │ immich_machine_learning | │ 220 │ │ except C.EPFail as err: │ immich_machine_learning | │ 221 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 222 │ │ │ │ print(f"EP Error: {str(er │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero immich_machine_learning | status code returned while running immich_machine_learning | OpenVINO-EP-subgraph_2 node. immich_machine_learning | Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap immich_machine_learning | h_2_0' Status Message: immich_machine_learning | /home/onnxruntimedev/onnxruntime/onnxruntime/core/p immich_machine_learning | roviders/openvino/ov_interface.cc:53 immich_machine_learning | onnxruntime::openvino_ep::OVExeNetwork immich_machine_learning | onnxruntime::openvino_ep::OVCore::LoadNetwork(const immich_machine_learning | string&, std::string&, ov::AnyMap&, std::string) immich_machine_learning | [OpenVINO-EP] Exception while Loading Network for immich_machine_learning | graph: immich_machine_learning | OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0C immich_machine_learning | heck 'false' failed at immich_machine_learning | src/inference/src/core.cpp:149: immich_machine_learning | invalid external data: immich_machine_learning | ExternalDataInfo(data_full_path: immich_machine_learning | Constant_2611_attr__value, offset: 0, data_length: immich_machine_learning | 0) immich_machine_learning | immich_machine_learning | immich_microservices | [Nest] 7 - 05/15/2024, 7:52:52 AM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | [Nest] 7 - 05/15/2024, 7:52:52 AM ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:86:31) immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 7 - 05/15/2024, 7:52:52 AM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "3f326640-2542-47db-b658-66543898ba07" immich_microservices | }

Snuupy commented 1 month ago

hello, using snuppy trick did the job for facial recognition however it still stands errors on smart search :

@jsapede

can you try again? made a change on Dockerfile, looks like I missed a line

https://github.com/Snuupy/immich/pkgs/container/immich-machine-learning/216456671?tag=v1.105.1-openvino is up

jsapede commented 1 month ago

not sur if i understand how to test ^^

Snuupy commented 1 month ago

@jsapede change your docker-compose.yml under immich-machine-learning

    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino to
    image: ghcr.io/snuupy/immich-machine-learning:v1.105.1-openvino
jsapede commented 1 month ago

ok will have a test tonight

jsapede commented 1 month ago

immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) immich_microservices | at async SearchRepository.upsert (/usr/src/app/dist/repositories/search.repository.js:188:9) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:91:9) immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "5c3721ca-a64d-4ce2-83a9-fdc15b54f424" immich_microservices | } immich_microservices | immich_postgres | 2024-05-15 11:12:14.288 UTC [65] ERROR: pgvecto.rs: The given vector is invalid for input. immich_postgres | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_postgres | 2024-05-15 11:12:14.288 UTC [65] STATEMENT: INSERT INTO "smart_search"("assetId", "embedding") VALUES ($1, '[-0.00472641,0.05822754,-0.023330688,0.029434204,-0.044403076,-0.011169434,0.051818848,0.002779007,0.04650879,0.017105103,0.010429382,0.008491516,-0.050964355,-0.0010976791,0.005554199,-0.001657486,-0.038330078,0.04360962,0.014129639,-0.043060303,0.07965088,0.020858765,0.022628784,-0.07476807,-0.017929077,-0.020385742,0.0284729,0.035064697,-0.00068330765,0.025421143,0.01612854,0.053527832,-0.0026435852,-0.0013608932,0.023208618,0.0027942657,-0.008598328,0.0051612854,0.019424438,0.084228516,0.02394104,0.022583008,-0.036010742,0.015899658,0.03225708,-0.14123535,-0.02772522,0.020217896,-0.008743286,-0.005756378,0.024810791,-0.020217896,0.020324707,0.0033988953,-0.023208618,0.033691406,-0.00749588,0.0011930466,0.01374054,-0.034851074,-0.048065186,-0.024459839,0.010444641,0.030822754,0.004508972,-0.021942139,0.033996582,0.1083374,0.0110321045,-0.035827637,-0.0021896362,-0.021728516,-0.018951416,-0.055725098,0.0055770874,-0.038513184,-0.010269165,0.0034484863,-0.012397766,-0.031219482,-0.012359619,-0.009246826,0.012069702,-0.016830444,0.033203125,-0.010108948,0.01209259,0.01386261,-0.00075006485,-0.051696777,0.010604858,-0.0041618347,-0.6850586,0.0138168335,-0.008117676,0.021026611,0.011917114,0.00022399426,-0.03262329,-0.10681152,-0.0060424805,-0.010658264,-0.008338928,0.013244629,0.020736694,0.021316528,-0.06512451,0.0107040405,0.0013160706,0.008262634,-0.029266357,-0.027328491,-0.052215576,-0.006549835,-0.018753052,0.012001038,-0.012680054,0.036834717,0.04373169,0.030593872,0.03756714,-0.031036377,-0.055908203,-0.013931274,0.017715454,-0.0008006096,-0.0012874603,0.0026359558,-0.0019264221,0.03286743,0.009559631,0.008552551,-0.0021438599,0.08544922,-0.015174866,0.027923584,0.021194458,-0.027862549,-0.017837524,-0.006389618,0.012397766,0.013069153,-0.0010528564,0.013214111,-0.022491455,-0.005317688,-0.053619385,0.025314331,-0.019546509,0.010269165,0.020004272,0.016921997,0.045654297,-0.042816162,0.031036377,-0.011940002,0.0023460388,-0.03842163,0.059051514,-0.010475159,-0.054382324,-0.035308838,-0.0024986267,0.023513794,-0.006137848,0.0022735596,0.044830322,0.03591919,0.002620697,-0.0032539368,0.0015945435,0.012001038,-0.03353882,-0.021606445,-0.0069999695,-0.015075684,-0.009689331,0.0038013458,-0.07434082,-0.0012989044,0.02355957,-0.0033931732,-0.03579712,0.000954628,0.012786865,-0.024765015,0.008148193,0.012756348,-0.0028266907,-0.012832642,-0.0060768127,-0.014297485,0.029891968,-0.00020158291,-0.078552246,-0.031280518,0.042938232,-0.008842468,0.010803223,-0.01586914,0.002986908,-0.011833191,0.0115356445,0.014930725,0.0009407997,-0.0018815994,-0.04168701,0.01701355,-0.018630981,0.05105591,-0.08270264,0.023925781,0.034301758,0.029724121,0.00020718575,-0.024505615,0.025131226,0.014511108,0.06695557,-0.013328552,-0.021224976,0.07763672,-0.0725708,-0.041625977,-0.0005373955,0.03286743,-0.037963867,0.0001399517,-0.0026435852,0.016723633,-0.05517578,-0.021636963,-0.00907135,-0.03503418,-0.028213501,-0.008598328,-0.017425537,-0.018096924,-0.018005371,-0.0021629333,0.022979736,-0.026779175,0.022659302,-0.030410767,-0.033813477,-0.023254395,0.008255005,0.016021729,-0.03201294,-0.011703491,-0.039520264,0.013671875,0.042907715,-0.0026798248,-0.028137207,0.013076782,-0.05496216,-0.01776123,0.16101074,0.03363037,-0.029525757,0.013900757,0.015037537,0.010681152,-0.015991211,-0.045562744,-0.011459351,0.04196167,0.014511108,-0.03967285,0.005039215,0.018447876,-0.03213501,-0.00409317,0.017608643,0.018585205,0.008155823,-0.015670776,-0.051330566,-0.020141602,0.015899658,-0.0003080368,-0.07092285,0.00058317184,0.006713867,0.022232056,-0.11553955,-0.01423645,-0.032928467,-0.026748657,-0.0037460327,-0.0034217834,-0.025939941,0.020507812,-0.030227661,0.006690979,0.1005249,-0.015808105,0.010948181,-0.0008792877,-0.011711121,0.0023117065,-0.01737976,0.0038661957,0.021224976,-0.0044136047,-0.022323608,-0.00491333,0.021148682,0.04019165,0.004234314,-0.015014648,0.08526611,-0.010047913,0.016784668,0.0048713684,0.027999878,0.055999756,-0.009315491,-0.08337402,0.021362305,0.16870117,-0.009353638,-0.038208008,0.016937256,-0.002922058,0.025909424,-0.0046691895,-0.036987305,0.032104492,-0.015541077,0.0109939575,-0.0032138824,-0.03878784,-0.03414917,-0.0054359436,0.009765625,0.035308838,0.027374268,0.010604858,-0.023605347,-0.031677246,-0.031173706,-0.007675171,-0.024642944,0.0065727234,-0.022827148,-0.008056641,-0.012542725,0.0049591064,0.028625488,-0.0032634735,0.07757568,-0.0029850006,-0.016555786,-0.03186035,-0.020828247,-0.066589355,0.015060425,0.011962891,-0.022323608,-0.017425537,-0.014533997,-0.027542114,-0.09362793,-0.02532959,0.0031356812,-0.066589355,0.0008621216,0.014961243,-0.028930664,0.010726929,0.019302368,-0.02947998,-0.013313293,0.012924194,0.029342651,-0.01725769,-0.0062828064,0.011276245,0.0011253357,-0.008880615,-0.025482178,0.0018310547,0.019363403,0.018859863,0.032287598,0.014457703,-0.039276123,0.105773926,-0.09094238,-0.03074646,0.02015686,-0.008270264,-0.0034713745,-0.008834839,0.05670166,0.027496338,-0.038085938,0.014419556,0.012298584,-0.036193848,0.0054016113,0.012512207,0.002822876,-0.0039863586,-0.0034999847,-0.0034828186,0.070251465,-0.042541504,-0.026657104,0.00856781,-0.0009965897,0.016174316,-0.034423828,0.018356323,0.004737854,-0.008659363,-0.042114258,0.0039749146,-0.019195557,-0.014984131,0.0049095154,-0.04345703,-0.0115356445,-0.004966736,-0.035217285,-0.021820068,-0.017868042,0.03717041,0.024307251,0.033447266,0.04650879,-0.002603531,-0.0231781,0.0026435852,-0.018325806,0.015289307,-0.019256592,-0.008934021,0.004043579,0.009521484,-0.0023288727,-0.031829834,-0.036193848,0.02017212,0.011253357,-0.024719238,-0.006023407,0.0030002594,-0.043792725,-0.023361206,-0.014152527,0.0020446777,0.026657104,-0.006462097,0.011756897,-0.019943237,-0.004043579,0.013961792,0.010429382,0.013305664,-0.045959473,-0.020599365,0.03186035,0.014312744,0.00774765,0.0019035339,-0.03942871,0.0012712479,0.019638062,0.009353638,0.015556335,-0.029830933,0.025390625,-0.053131104,-0.0037784576,-0.024429321,0.010353088,0.0088272095,0.008308411,-0.024505615,0.024719238,0.044830322,0.019927979,-0.015991211,0.011962891,0.01448822,0.0059127808,-0.02658081,0.011604309,-0.0008454323,-0.00381279,-0.07067871,-0.006931305,-0.02557373,0.008361816,0.016799927,-0.025466919,-0.011802673,0.025024414,-0.0066566467,-0.0046844482,0.004814148,0.027282715,-0.010559082,0.020477295,-0.009292603,-0.005634308,0.103759766,-0.037750244,-0.014671326]') ON CONFLICT ( "assetId" ) DO UPDATE SET "assetId" = EXCLUDED."assetId", "embedding" = EXCLUDED."embedding" immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) immich_microservices | at async SearchRepository.upsert (/usr/src/app/dist/repositories/search.repository.js:188:9) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:91:9) immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 7 - 05/15/2024, 11:12:14 AM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "4ee62067-827d-4a1e-aa0c-be0ec78e4833" immich_microservices | } immich_microservices | immich_postgres | 2024-05-15 11:12:14.368 UTC [65] ERROR: pgvecto.rs: The given vector is invalid for input. immich_postgres | ADVICE: Check if dimensions and scalar type of the vector is matched with the index.

jsapede commented 1 month ago

with ViT-B-32__openai

facial recognition works

Snuupy commented 1 month ago

after changing the line in docker-compose.yml did you docker compose down && docker compose up -d?

my logs show no errors upon uploading.

jsapede commented 1 month ago

yes, i did it, dont know where the error is coming form

looks like this : https://github.com/immich-app/immich/discussions/7381

but there seems to be no error on permissions in database.

Snuupy commented 1 month ago

does that happen on a new/separate instance of immich?

jsapede commented 1 month ago

looks better like this : https://github.com/immich-app/immich/discussions/7425 trying to figure how to re-compute search index

jsapede commented 1 month ago

does that happen on a new/separate instance of immich?

https://github.com/immich-app/immich/discussions/7425#discussioncomment-9446357

seems solved !

image

ilijamt commented 1 month ago

Works with ghcr.io/snuupy/immich-machine-learning:v1.105.1-openvino for me.

LeoAdL commented 1 month ago

It fixes the face detection (thank you @Snuupy!), but then the smart search is not working (while it is the reverse with the current open-vino release).

Snuupy commented 1 month ago

@LeoAdL

It fixes the face detection (thank you @Snuupy!), but then the smart search is not working (while it is the reverse with the current open-vino release).

can you check if it's this issue? https://github.com/immich-app/immich/discussions/7425#discussioncomment-9446151

LeoAdL commented 1 month ago

I am trying to rerun again, but for the moment I get unrelated results and the following error message:

immich_server            | [Nest] 7  - 05/19/2024, 5:57:56 PM    WARN [ImmichServer] [ExpressAdapter] Content-Type doesn't match Reply body, you might need a custom ExceptionFilter for non-JSON responses
jsapede commented 1 month ago

Look at my logs upper in the post to see if its the same pb

LeoAdL commented 1 month ago

I do not get any error messages anymore, but the results are non-relevant as in pre 1.99.0. It works perfectly fine with the CPU loading the same model.

Snuupy commented 1 month ago

I do not get any error messages anymore, but the results are non-relevant as in pre 1.99.0. It works perfectly fine with the CPU loading the same model.

try again with a new instance, does this still happen? you don't have to delete your data but tell me if it works on a fresh instance.

also pretty sure this is an openvino (upstream) issue not an immich issue

jsapede commented 1 month ago

I do not get any error messages anymore, but the results are non-relevant as in pre 1.99.0. It works perfectly fine with the CPU loading the same model.

I'll have a try tomorrow with CPU as i've also observed irrevelency

Screenshot_20240519-210505.png

Although some searches looks cohérent

Screenshot_20240519-210613.png

Snuupy commented 1 month ago

you may want to try a better model, check on discord what people are using but I haven't done much research into it

the default one iirc is not that great but I think it was done because not everyone has beefy hardware

mertalev commented 1 month ago

If it worked well on CPU, but not on OpenVINO, then it indicates a bug in how the model was compiled to OpenVINO format.

jsapede commented 1 month ago

ok, tried this morning with another CLIP multilingual model :

https://huggingface.co/immich-app/nllb-clip-large-siglip__v1

got the following errors :

`immich_machine_learning | 2024-05-20 07:06:14.817966319 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running OpenVINO-EP-subgraph_3 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_3_0' Status Message: /home/onnxruntimedev/onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:53 onnxruntime::openvino_ep::OVExeNetwork onnxruntime::openvino_ep::OVCore::LoadNetwork(const string&, std::string&, ov::AnyMap&, std::string) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_3_0Check 'false' failed at src/inference/src/core.cpp:149: immich_machine_learning | invalid external data: ExternalDataInfo(data_full_path: Constant_1863_attr__value, offset: 0, data_length: 0) immich_machine_learning | immich_machine_learning | immich_machine_learning | [05/20/24 07:06:14] ERROR Exception in ASGI application
immich_machine_learning |
immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:118 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 115 │ │ immich_machine_learning | │ 116 │ model = await load(await model_cache. │ immich_machine_learning | │ ttl=settings.model_ttl, kwargs)) │ immich_machine_learning | │ 117 │ model.configure(kwargs) │ immich_microservices | [Nest] 7 - 05/20/2024, 7:06:14 AM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error immich_machine_learning | │ ❱ 118 │ outputs = await run(model.predict, in │ immich_microservices | [Nest] 7 - 05/20/2024, 7:06:14 AM ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │ immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:86:31) immich_machine_learning | │ 120 │ immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_machine_learning | │ 121 │ immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_machine_learning | │ │ immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_machine_learning | │ /usr/src/app/main.py:125 in run │ immich_microservices | [Nest] 7 - 05/20/2024, 7:06:14 AM ERROR [ImmichMicroservices] [JobService] Object: immich_machine_learning | │ │ immich_microservices | { immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │ immich_microservices | "id": "aeffa9af-b9af-4bba-9a62-eb939a1b5a28" immich_machine_learning | │ 123 │ if thread_pool is None: │ immich_machine_learning | │ 124 │ │ return func(inputs) │ immich_microservices | } immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │ immich_microservices | immich_machine_learning | │ 126 │ immich_machine_learning | │ 127 │ immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:59 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 56 │ │ self.load() │ immich_machine_learning | │ 57 │ │ if model_kwargs: │ immich_machine_learning | │ 58 │ │ │ self.configure(**model_kwargs │ immich_machine_learning | │ ❱ 59 │ │ return self._predict(inputs) │ immich_machine_learning | │ 60 │ │ immich_machine_learning | │ 61 │ @abstractmethod │ immich_machine_learning | │ 62 │ def _predict(self, inputs: Any) -> An │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/clip.py:52 in _predict │ immich_machine_learning | │ │ immich_machine_learning | │ 49 │ │ │ case Image.Image(): │ immich_machine_learning | │ 50 │ │ │ │ if self.mode == "text": │ immich_machine_learning | │ 51 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ ❱ 52 │ │ │ │ outputs: NDArray[np.float │ immich_machine_learning | │ self.transform(image_or_text))[0][0] │ immich_machine_learning | │ 53 │ │ │ case str(): │ immich_machine_learning | │ 54 │ │ │ │ if self.mode == "vision": │ immich_machine_learning | │ 55 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:21 │ immich_machine_learning | │ 9 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 216 │ │ if not output_names: │ immich_machine_learning | │ 217 │ │ │ output_names = [output.name f │ immich_machine_learning | │ 218 │ │ try: │ immich_machine_learning | │ ❱ 219 │ │ │ return self.sess.run(output │ immich_machine_learning | │ 220 │ │ except C.EPFail as err: │ immich_machine_learning | │ 221 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 222 │ │ │ │ print(f"EP Error: {str(er │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero
immich_machine_learning | status code returned while running
immich_machine_learning | OpenVINO-EP-subgraph_3 node.
immich_machine_learning | Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap immich_machine_learning | h_3_0' Status Message:
immich_machine_learning | /home/onnxruntimedev/onnxruntime/onnxruntime/core/p immich_machine_learning | roviders/openvino/ov_interface.cc:53
immich_machine_learning | onnxruntime::openvino_ep::OVExeNetwork
immich_machine_learning | onnxruntime::openvino_ep::OVCore::LoadNetwork(const immich_machine_learning | string&, std::string&, ov::AnyMap&, std::string)
immich_machine_learning | [OpenVINO-EP] Exception while Loading Network for immich_machine_learning | graph:
immich_machine_learning | OpenVINOExecutionProvider_OpenVINO-EP-subgraph_3_0C immich_machine_learning | heck 'false' failed at
immich_machine_learning | src/inference/src/core.cpp:149:
immich_machine_learning | invalid external data:
immich_machine_learning | ExternalDataInfo(data_full_path:
immich_machine_learning | Constant_1863_attr__value, offset: 0, data_length: immich_machine_learning | 0)
immich_machine_learning |
immich_machine_learning |
immich_machine_learning | mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x75043c020000 immich_machine_learning | (this may still be a valid very large allocation (over 64MiB)) immich_machine_learning | mimalloc: warning: (yes, the previous pointer 0x75043c020000 was valid after all) immich_machine_learning | mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x75043c020000 immich_machine_learning | (this may still be a valid very large allocation (over 64MiB)) immich_machine_learning | mimalloc: warning: (yes, the previous pointer 0x75043c020000 was valid after all)

switching to CPU version generates index errors :

immich_postgres | 2024-05-20 07:12:39.522 UTC [58] ERROR: pgvecto.rs: The given vector is invalid for input. immich_postgres | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_postgres | 2024-05-20 07:12:39.522 UTC [58] STATEMENT: INSERT INTO "smart_search"("assetId", "embedding") VALUES ($1, '[-0.0019855415,0.010838846,-0.029174354,0.002818098,0.012467433,0.008725432,0.041027203,0.039598167,0.010343627,0.010736328,0.031523224,0.017317837,0.00687775,0.026798358,-0.0063249413,0.004649172,0.02930324,0.007233265,0.0068142363,-0.032617845,-0.002749845,0.038012993,0.0071095475,0.0057572112,0.0054148557,-0.008628646,0.010526898,0.003930114,0.023705563,-0.0020050213,0.015157387,0.032650843,0.059936408,-0.0104608135,0.01572758,-0.005988058,-0.0028063743,0.029138738,0.013989555,-0.075763725,-0.0044368384,0.07258743,-0.020399388,-0.031316787,-0.015580373,-0.0067963796,-0.0035901598,0.025561325,-0.014757424,-0.05069377,0.008522092,0.0043534483,0.013662222,0.0097612655,0.043051407,0.01094251,0.0034714253,0.0027126786,-0.015136938,0.02098717,0.0040651504,0.012211872,-0.002641023,0.01159998,0.0012533838,0.0075528813,-0.0014564259,0.095892,0.0045336457,0.03142923,0.01874761,-0.016212484,0.019141575,0.009574559,0.005998863,0.0036997248,-0.003390201,0.013179829,-0.0019859069,0.00008743155,-0.016983982,-0.06334343,0.017062837,-0.02066563,0.00075131096,0.014623942,-0.031834573,-0.10912981,0.010094315,-0.00094148336,0.019950747,0.018292136,0.008355061,0.043532126,-0.018467402,-0.004111094,0.08437387,0.0027382867,0.005660722,-0.017966252,0.020790385,0.017051019,-0.005805263,0.024290001,0.0004450746,0.041904084,-0.0029454057,-0.02039585,-0.0026057505,0.0045338366,-0.086000465,0.0028620837,0.010336116,0.04469745,-0.0387677,0.04253522,0.001593244,0.0036192918,-0.005555969,0.016839882,0.0077302004,0.011214501,-0.00075549143,-0.020474417,-0.01643536,-0.00059499696,0.008017416,-0.015769467,-0.0052264216,0.013289356,-0.03897138,0.0422748,0.032295126,-0.03729524,0.01576651,-0.021633899,-0.013320596,-0.0042101,0.009985446,0.027664343,-0.0040467223,-0.017362136,0.007229458,-0.007603551,0.003070253,-0.033369754,0.013649768,-0.009320836,0.017493516,-0.025754659,0.0082945675,0.0037929425,-0.019173807,0.0002886317,0.016121626,-0.012089363,-0.002669867,0.016485445,-0.0057186075,-0.058869135,0.04214346,0.031347718,-0.0036423467,-0.053850126,-0.029839393,0.03230932,-0.014139785,0.0079337545,0.015898073,-0.011144386,0.01474961,-0.0056539634,0.035507325,-0.01607546,-0.027144708,-0.021508971,0.0043789763,0.0065087234,0.005539463,-0.01708434,-0.025535328,-0.004875438,0.033150624,-0.00016492666,0.009788488,-0.009006273,0.0019387872,0.0094013335,-0.02090326,-0.009554029,-0.0009662287,-0.03966385,-0.03451743,-0.0035995597,-0.00935717,-0.00077361736,-0.0006808785,-0.015186684,-0.0024114607,-0.05129077,0.030693313,-0.011788347,-0.010407987,0.008432299,0.079326704,-0.019320562,0.0025003608,-0.010751247,0.041894834,-0.06277741,0.03159925,-0.027415007,0.0028766894,-0.0068278587,0.04876738,-0.027828421,-0.027831417,0.013794709,-0.020804487,0.0590946,-0.005588765,0.0077075968,-0.033997837,-0.0015053883,0.0071494603,-0.0050133406,0.015988147,-0.05031242,-0.0021751712,-0.003974665,0.050540894,0.012233619,-0.0010147277,0.005869418,0.03888606,0.010319565,0.012629805,0.015805218,0.002629758,0.015049306,0.009385251,0.014307854,-0.037643004,0.015515373,0.0075011835,0.0046999464,-0.00756627,0.015629565,0.00028395385,0.050220154,0.03239413,-0.03155103,-0.007795341,-0.02563749,-0.012203387,-0.021420198,-0.01414403,0.011344997,-0.0018497485,-0.0013184062,-0.024465786,0.013881047,-0.0063489936,-0.017382024,0.05370967,0.02178771,-0.0375393,0.0018370441,-0.01556864,0.01919757,-0.022377754,0.030400733,0.015316274,-0.008547495,0.014570179,0.023167362,0.016621256,-0.020938328,0.030121246,0.011552597,0.008345348,-0.014588702,0.031437263,0.026334273,-0.008596759,-0.001444259,-0.018005095,0.050699256,-0.0041018357,0.0210611,-0.002586978,0.0030559578,0.03960314,0.051853005,-0.030750772,0.0009304943,0.02007747,-0.01914861,0.009221662,-0.011844958,-0.0050868727,-0.012257818,-0.0023580468,0.0052637933,0.003971098,-0.011201618,-0.03651185,0.013589299,0.0025933364,0.00019741432,0.007132763,0.023466192,-0.011418806,-0.009989008,0.020961419,-0.0060674064,-0.0021419073,0.044515833,-0.012813107,0.0031286299,-0.027216032,0.0011037278,-0.023690823,-0.017354487,-0.008969352,0.025557933,-0.016412672,0.0014773556,0.013555329,-0.036769107,-0.033157703,0.015060061,0.030128136,0.004839296,-0.028392915,-0.00010243815,-0.0021990468,0.014580072,-0.013095612,-0.01173983,-0.026444593,-0.021609446,0.004220385,-0.015479424,0.0036506874,0.0039608246,0.0016100337,-0.014807571,0.026595546,0.020583823,-0.0023464367,-0.023112163,0.041082457,0.059100043,0.029750377,0.019056499,-0.00002973981,0.024933511,0.012403567,-0.018697515,-0.012768128,0.014109888,-0.0064278534,0.0004889159,0.009531707,-0.014812875,0.024859415,-0.005187958,-0.0016208459,-0.0552704,-0.0058529233,-0.011052697,-0.019483171,-0.0070474297,0.020129908,-0.035629906,0.018111331,-0.009180377,0.043751616,0.005402257,-0.03844198,0.0031179388,0.0042051696,-0.04101828,0.016941393,-0.09834079,-0.03573986,-0.022560697,0.008047459,-0.024356378,0.0032567084,0.0058812476,-0.010985473,-0.030389253,-0.006831017,-0.016766278,0.33248463,-0.0024098197,0.01981675,0.013583864,0.01136825,-0.00068603444,-0.022215448,-0.012830227,0.0062303278,-0.021714527,-0.0026283017,0.006174015,0.021479784,0.006014162,-0.01222629,-0.019272713,-0.03524566,0.031139849,-0.00009062656,0.00445403,-0.0037642,0.002388128,-0.00025483497,0.00372654,0.011396343,-0.05726363,-0.030236969,0.042430814,0.017122932,-0.014131476,0.009849315,0.03684108,-0.0066480846,0.0030170036,0.025365744,0.011301708,0.00022852921,-0.005817174,0.027666176,-0.00945315,0.048664004,-0.0028992314,0.037292548,-0.00812772,-0.03273666,-0.016255891,0.005398753,0.0005858483,-0.03569707,-0.0037628922,-0.020592779,0.032285035,-0.008978221,0.004336717,0.00037140318,-0.008444069,-0.0017066873,-0.00857066,0.017406391,-0.030746143,0.006580784,0.025374334,-0.0017056431,0.029419748,0.027111048,-0.0067858803,0.025051305,-0.00217508,0.010886521,0.015165656,0.0027649342,0.04527392,0.0050032404,0.007929213,0.002127859,0.024426224,-0.011367906,0.0045012115,0.013269973,-0.0011541636,-0.003674581,-0.0042461357,0.017718995,-0.016212448,-0.011162395,-0.012410965,0.07438335,0.009460903,0.053563196,0.018185418,-0.0036409183,0.0056124786,-0.020877788,-0.012248509,-0.055684224,-0.0300063,0.005605591,0.0015743672,0.021530757,0.012835346,-0.0062871757,0.030446287,0.029923828,0.018758412,0.006240113,-0.030803323,-0.024015145,-0.114027254,-0.0041212183,-0.02119994,-0.015214744,0.0048919255,-0.008787689,0.023787493,-0.010822425,0.024513079,-0.02552909,-0.00021786489,0.021573847,0.021876551,-0.0066973604,0.051443584,0.022484904,0.009542761,-0.0032796643,-0.012736042,0.124713644,-0.0018544082,0.006963596,0.0043830182,0.036936592,0.029415507,-0.0052262736,0.039878357,-0.0012797015,-0.035017464,-0.013877981,0.010278876,0.0035716223,-0.0060644513,0.0019228847,0.023729647,-0.0018261763,0.005952108,0.034911454,-0.009732786,-0.0064176023,-0.01423805,-0.0112532275,-0.022168238,-0.013493341,0.0061974404,-0.010736995,-0.009337253,0.011849059,0.01830878,0.022412112,-0.031821035,0.012104872,0.017608253,-0.00866594,-0.009600198,0.0031048986,-0.0005747472,0.022618392,0.007238528,-0.017283961,0.0075308396,0.02807254,0.015929192,0.31723925,0.0038892555,-0.103820205,0.010800004,0.007386441,-0.009709718,-0.0141321,-0.007351959,0.02072022,-0.0677206,0.008712263,-0.0066603674,0.010993041,-0.015734572,-0.0073447987,-0.03218226,-0.012194646,0.07605815,-0.047259584,-0.02445212,-0.017833328,0.0013917293,-0.0023584994,0.0028352246,0.029045602,0.017222086,0.004918675,-0.0011489338,0.0036829265,0.017196165,-0.0014529142,0.01026802,0.0024675794,-0.026431564,0.015737755,0.009225022,0.040628072,0.05012866,-0.017548013,-0.020898782,0.0025738652,0.010931237,0.03094857,0.00023174778,0.009138983,-0.0030948494,-0.002452989,-0.025852248,0.01108547,0.010421942,-0.023611333,0.028700346,0.0294027,-0.011055125,0.046073515,-0.016093485,-0.014824992,0.0031906914,-0.009984209,0.01912326,-0.022554178,-0.0071835,-0.012974766,-0.016929094,0.0068614255,0.0020106207,-0.024374688,-0.0071932683,-0.0067832917,-0.050707683,0.033923425,-0.0060804286,0.0197791,0.0054301675,0.0110023655,-0.012495113,0.014088298,0.01843656,-0.009256857,0.039587483,0.010218777,-0.04277436,0.015271311,0.008310979,-0.03712614,-0.00043103108,-0.06316297,-0.03997194,-0.055672318,0.04985065,-0.00040622242,0.029664451,0.016059522,-0.008463222,-0.023397753,-0.023563145,0.0021523999,0.0040681246,0.019215468,-0.037239652,0.025408521,-0.015296172,0.015302598,-0.07876884,0.014149577,0.008808588,0.021324381,0.007530429,-0.048694264,-0.008494259,-0.023488265,0.024470571,-0.016983455,-0.019496925,-0.03473102,0.014313353,-0.02368762,-0.00059865613,-0.021394392,-0.013052518,-0.033786807,-0.03228076,0.022011679,-0.01085115,-0.037880417,0.019438798,-0.019048115,-0.024016066,0.048220553,-0.02274599,0.0054148803,0.0027644956,-0.022523694,0.04519563,-0.03053855,0.026470136,0.018415751,0.028542724,0.008998034,-0.010272362,-0.052127205,0.05657615,0.005865801,-0.06404257,0.0025329709,-0.0015777576,-0.005284834,0.01760826,0.01465583,-0.0062150755,0.025058042,-0.0010952032,-0.03837947,0.022024784,0.022714207,-0.024965791,-0.0056510023,0.019675009,0.018327897,-0.030646542,0.009492211,0.012657979,-0.010659515,-0.07485447,0.039064214,0.0027297782,-0.004984802,-0.043293305,-0.02847156,0.041495137,0.04545222,0.05343759,0.0061942358,-0.0026678846,0.015734177,-0.008088579,0.004124958,-0.013294719,0.014308398,-0.055976205,0.02426776,-0.007714878,0.005053037,-0.01151765,-0.0694673,0.012390375,-0.025953842,-0.022536218,0.024847394,-0.033472992,-0.027547095,-0.07228807,-0.011319938,-0.029956512,0.016977932,0.035454597,0.015868539,-0.018978506,-0.043153275,-0.01257776,0.0062007704,0.05414892,-0.015917348,0.025495809,0.021004898,-0.017054819,-0.0025748906,-0.033784878,-0.0018346407,0.0059578256,0.009166321,0.011057435,-0.0010295374,0.0070021367,0.013235237,-0.006476334,0.00020144728,-0.044548254,0.009601461,0.0012201972,0.019678043,0.006111603,-0.004127794,-0.10340391,0.014938963,0.004310968,0.013426923,0.008428755,0.011666366,0.03171064,0.025008556,0.0035368656,-0.011679063,-0.0068322006,-0.006447802,-0.0123948315,0.001378896,0.0021583396,0.004250862,-0.0018121706,0.07658485,0.008247042,0.0012884887,-0.017804168,0.026834369,-0.03385441,-0.003047365,-0.061322827,-0.0053489506,-0.009447964,0.031760003,0.0037417712,-0.002796092,0.030887982,0.015285987,0.003847884,-0.017474221,0.035982825,0.039007504,-0.01005999,-0.01049281,-0.016585538,-0.014911385,0.009746717,-0.02634134,0.023116492,-0.009173959,-0.023854068,0.017088765,0.02932745,0.019920208,-0.03215282,0.0054006646,0.004158479,-0.03832187,-0.0013739378,-0.023113145,0.004753121,0.01724855,0.009596302,0.018563986,0.0042409888,0.023288043,-0.014476339,-0.008139111,-0.0147378165,-0.0022294847,0.015614078,0.05278061,0.009716018,0.0015020813,0.0013666898,0.0088598095,0.0039072987,-0.04328564,0.02602132,0.00010645258,0.0039948407,-0.010181326,-0.016185315,0.024533778,0.0021075876,0.0020711042,-0.031021118,0.000006020867,0.012494586,-0.014848178,-0.021723907,0.005257161,-0.02135788,-0.000010206359,-0.017712899,0.019915465,0.0073195533,-0.010014445,0.023491839,-0.007146822,-0.010172202,0.029203193,0.014645069,-0.005990033,0.040127967,0.022012006,-0.03355523,-0.016933458,0.026954716,0.08390037,0.0010129453,-0.003216249,-0.05211232,0.0172377,0.013310515,-0.010129626,0.014966631,0.01532531,-0.0009873072,0.021938484,0.0067835804,0.012847664,-0.0057387482,0.026056787,0.011638377,0.028129505,-0.00070107426,0.003216312,0.00088604365,0.033320747,0.013601675,-0.01513385,-0.02065974,-0.022391235,-0.014603591,-0.006107612,-0.02089128,0.027280636,0.0044668363,0.04695085,0.028469076,0.0061978726,0.055290233,-0.04284477,-0.016095638,-0.009593353,-0.029948063,0.0076148687,-0.0003708919,0.0038001118,0.010611897,0.061950527,-0.009345062,0.038630374,-0.015583447,0.014732325,0.010095144,-0.0006472529,0.016310578,0.01692941,-0.019127468,-0.025066866,-0.011416806,0.023294728,0.027012054,0.014196692,-0.008061255,0.024433045,0.021820158,-0.043486368,0.07964974,0.0029539012,-0.019773016,-0.03797377,0.008308019,-0.0020944988,-0.035489995,0.21483718,-0.010081883,-0.02057748,-0.0028068163,0.07584366,-0.01907087,-0.010922338,-0.015625792,0.021903338,-0.002243903,0.019166974,-0.10204007,-0.06859482,-0.06722186,0.012359549,0.009790601,-0.08228961,-0.038346637,-0.042886492,-0.027809987,-0.011207424,-0.0027738714,-0.018866023,-0.048638143,0.013831542,-0.018182462,-0.024480185,0.019822802,0.0047991658,0.056335084,0.016223602,-0.013279018,-0.02778767,0.03304153,-0.015032859,0.018413268,0.025027517,0.012795506,-0.038439743,0.049985114,0.029315794,-0.0077698287,0.0052026967,0.0050503057,0.005666021,-0.013828864,-0.015155823,-0.0075423927,0.01669791,0.02087168,0.030162003,0.025122603,0.00084190123,0.059500836,-0.026433961,-0.037963554,-0.018264491,-0.034659255,-0.009608632,0.004603804,0.02116765,0.08370657,-0.028065113,-0.008858235,0.015707046,-0.002496879,0.009077163,0.022341302,0.011124382,-0.03372358,0.018560076,0.007467675,0.03185154,0.00012686761,0.028106594,0.018324751,0.0035599358,-0.02282597,0.026226135,-0.030925076,0.023414025,-0.0074871955,0.015916003,-0.0055549513,0.023719592,-0.012383069,-0.00029759502,0.01979294,0.027806683,0.008795049,-0.0069119185,0.037198983,0.01617551,-0.018703893,-0.01336036,-0.017518459,-0.024507985,0.01175815,-0.0035379755,-0.012808608,-0.04781528,0.012852902,-0.01773001,0.003361258,0.021779321,0.019678915,0.008854227,-0.030475082,-0.03115345,0.009938747,0.040895727,-0.005863072,0.030192694,0.0016849671,0.013721202,-0.0019523859,-0.006397204,-0.04769137,-0.002055337,-0.02697364,0.016426614,-0.03348636,-0.010341175,-0.0352914,0.02833678,-0.006802132,-0.036528613,0.018286847,0.0011856506,-0.03450767,-0.026124204,-0.008062548,-0.031746425,-0.018078893,-0.00031229426,-0.0041492195,0.006495626,0.015796948,-0.0179632,0.020635607,-0.012449595,0.0036386002,-0.00921202,-0.008889971,-0.012542233,0.010821392,0.010221491,-0.041698154,0.0025290267,-0.00038564444,0.0021938323,0.03493735,-0.016098024,-0.009125528,0.018473601,-0.02167627,-0.018372053,-0.04653088,0.008323465,0.015581499,-0.0047012684,-0.042111136,-0.04341909,0.031973097,-0.007426423,-0.007438647,0.0066369567,-0.02338152,-0.005253704,-0.018847538,0.012471524,-0.028666757,0.024627516,-0.0046333857,0.012337033,0.05235776,-0.023071734,0.0139986,0.026481751,0.021908706,-0.0149647575,0.00029280677,-0.008864358,0.010789001,-0.02068297,-0.009283273,-0.046355736,0.00017902735,0.011119871,-0.031419925,-0.02888088,-0.0065652924]') ON CONFLICT ( "assetId" ) DO UPDATE SET "assetId" = EXCLUDED."assetId", "embedding" = EXCLUDED."embedding" immich_microservices | [Nest] 6 - 05/20/2024, 7:12:39 AM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | [Nest] 6 - 05/20/2024, 7:12:39 AM ERROR [ImmichMicroservices] [JobService] QueryFailedError: pgvecto.rs: The given vector is invalid for input. immich_microservices | ADVICE: Check if dimensions and scalar type of the vector is matched with the index. immich_microservices | at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) immich_microservices | at async SearchRepository.upsert (/usr/src/app/dist/repositories/search.repository.js:188:9) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:91:9) immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 6 - 05/20/2024, 7:12:39 AM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "17874952-3342-437e-b0f2-ff401bbfe0d5" immich_microservices | } immich_microservices |

even applying https://github.com/immich-app/immich/discussions/7425#discussioncomment-9446151 trick

[EDIT] relaunched smart search CPU on "ALL" seems to compute now ... these machine learnings settings / are quite obscure especially if yuo dont have eye on logs to see if it works

[EDIT 2] CPU version of the https://huggingface.co/immich-app/nllb-clip-large-siglip__v1 works and gives reliable results however OpenVino version still returns errors as on top of this post.

jsapede commented 1 month ago

following the tests, rebuild immich instance from zero, using the simplest model : immich-app/RN50__openai

build immich with CPU smart search and it works and give reliable results. Then rebuilt immich with snuupy openvino and it also works and alos give reliable results

Done the same tests with anorther intermediate CLIP model : LABSE-Vit-L-14

Tried openVino First but failed (error) then switched to CPU and it works, and give reliable results

looks like some models are not available with openvino !