huggingface / optimum

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
https://huggingface.co/docs/optimum/main/
Apache License 2.0
2.45k stars 435 forks source link

ORTModelForCustomTasks lacks attributes #1992

Open TheMattBin opened 3 weeks ago

TheMattBin commented 3 weeks ago

System Info

python: 3.11
OS: Linux
torch: 2.4.0
optimum: 1.21.3
onnx: 1.16.2
onnxruntime: 1.18.1
onnxruntime-gpu: 1.19.0

Who can help?

@JingyaHuang @echarlaix

Information

Tasks

Reproduction (minimal, reproducible, runnable)

Hi, I'm running my customized onnx model with optimum with CUDA using ORTModelForCustomTasks. I can run my model with both CPU and Intel-Openvino. However, when I run the script below, it shows errors AttributeError: 'ORTModelForCustomTasks' object has no attribute 'inputs_names' which I assumed related to optimum/onnxruntime/io_binding/io_binding_helper.py:160. It would be helpful if you can provide some hints with this issue.

image_processor = AutoProcessor.from_pretrained("deta_onnx2_gpu")
model = ORTModelForCustomTasks.from_pretrained("deta_onnx2_gpu", provider="CUDAExecutionProvider")

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

inputs = image_processor(images=image, return_tensors="pt")
print(inputs.keys())
# assert model.providers == ["CUDAExecutionProvider", "CPUExecutionProvider"]
with torch.no_grad():
    outputs = model(**inputs)

Expected behavior

I would expect the code to be run and provide similar detection output like running on onnx model with CPU.

TheMattBin commented 3 weeks ago

I also tested with DETR (onnx with cuda) which is supported by optimum.cli, also got the same issue.

optimum-cli export onnx -m facebook/detr-resnet-50 --task 'object-detection' --framework 'pt' detr_gpu  --device cuda