search

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Apache License 2.0
1.91k stars 175 forks source link

Updating transformers issue with zero-shot-image-classification #540

Open loadams opened 3 weeks ago

loadams commented 3 weeks ago

Updating to transformers versions beyond v4.42.4 causes issues with the CI tests in the legacy mode. The zero-shot-image-classification tests fail with:

FAILED test_non_persistent_deployment.py::test_single_GPU[None-50050-False-28080-fp16-1-False-False-1-True-False-ds_config0-zero-shot-image-classification-openai/clip-vit-base-patch32-query6-non-persistent] - ValueError: Attempting to cast a BatchFeature to type None. This is not supported.

We have isolated the problematic commit to this one: https://github.com/huggingface/transformers/pull/31342

../../mii/legacy/client.py:144: in query
    return task_methods.run_inference(inference_pipeline, args, query_kwargs)
../../mii/legacy/method_table.py:255: in run_inference
    return inference_pipeline(image, candidate_labels=candidate_labels, **kwargs)
../../../venv/lib/python3.12/site-packages/transformers/pipelines/zero_shot_image_classification.py:107: in __call__
    return super().__call__(images, **kwargs)
../../../venv/lib/python3.12/site-packages/transformers/pipelines/base.py:1257: in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
../../../venv/lib/python3.12/site-packages/transformers/pipelines/base.py:1263: in run_single
    model_inputs = self.preprocess(inputs, **preprocess_params)
../../../venv/lib/python3.12/site-packages/transformers/pipelines/zero_shot_image_classification.py:124: in preprocess
    inputs = inputs.to(self.torch_dtype)