Subsequent requests hang if one failed for embeddings_image api

[kimson99]

System Info

OSVersion: Linux Platform: Docker image on Cloudrun GPU: NVIDIA L4 Current version: michaelf34/infinity:latest Model: openai/clip-vit-large-patch14

Information

• [X] Docker
• [ ] The CLI directly via pip

Tasks

• [ ] An officially supported command
• [ ] My own modifications

Reproduction

1. Call /embeddings_image with this as the body

   {
   "input": [
   "https://upload.wikimedia.org/wikipedia/commons/c/ca/1x1.png"
   ]
   }

2. Error log on the server infinity_emb ERROR: mean batch_handler.py:449 must have 1 elements if it is an iterable, got 3 Traceback (most recent call last): File "/app/infinity_emb/inference/batch_handler.py", line 432, in _preprocess_batch feat = self._model.encode_pre(items_for_pre) File "/app/infinity_emb/transformer/vision/torch_vision .py", line 88, in encode_pre preprocessed = self.processor( File "/app/.venv/lib/python3.10/site-packages/transform ers/models/clip/processing_clip.py", line 109, in __call__ image_features = self.image_processor(images, return_tensors=return_tensors, **image_processor_kwargs) File "/app/.venv/lib/python3.10/site-packages/transform ers/image_processing_utils.py", line 41, in __call__ return self.preprocess(images, **kwargs) File "/app/.venv/lib/python3.10/site-packages/transform ers/models/clip/image_processing_clip.py", line 340, in preprocess images = [ File "/app/.venv/lib/python3.10/site-packages/transform ers/models/clip/image_processing_clip.py", line 341, in <listcomp> self.normalize(image=image, mean=image_mean, std=image_std, input_data_format=input_data_format) File "/app/.venv/lib/python3.10/site-packages/transform ers/image_processing_utils.py", line 111, in normalize return normalize( File "/app/.venv/lib/python3.10/site-packages/transform ers/image_transforms.py", line 392, in normalize raise ValueError(f"mean must have {num_channels} elements if it is an iterable, got {len(mean)}") ValueError: mean must have 1 elements if it is an iterable, got 3
3. All subsequent requests are timeout

Expected behavior

Subsequent requests should be processed

[michaelfeil]

Generally if the server crashes with traceback, it is not restarted.

It seems, if the image is a 1x1 image, some tensor dimensions get collapsed. Making a (3x256x256) can be normalized over mean[1,2]-axis, while a (3x1x1) image might collapse into a (3,) tensor, which only has one axis.

A user request should not be able to kill the server. I'll look into it.