neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs
https://neuralmagic.com/deepsparse/
Other
2.94k stars 169 forks source link

How can I make proper request to server #1648

Open Alex-SSK opened 1 month ago

Alex-SSK commented 1 month ago

Describe the bug A clear and concise description of what the bug is.

Expected behavior A clear and concise description of what you expected to happen.

Environment Include all relevant environment information:

  1. OS [e.g. Ubuntu 18.04]:
  2. Python version [e.g. 3.8]:
  3. DeepSparse version or commit hash [e.g. 0.1.0, f7245c8]:
  4. ML framework version(s) [e.g. torch 1.7.1]:
  5. Other Python package versions [e.g. SparseML, Sparsify, numpy, ONNX]:
  6. CPU info - output of deepsparse/src/deepsparse/arch.bin or output of cpu_architecture() as follows:
    >>> import deepsparse.cpu
    >>> print(deepsparse.cpu.cpu_architecture())

To Reproduce Exact steps to reproduce the behavior:

Errors If applicable, add a full print-out of any errors or exceptions that are raised or include screenshots to help explain your problem.

Additional context Add any other context about the problem here. Also include any relevant files.

I want to deploy Yolov8 with DeepSparse Server. How can I properly make a request with images data?

the endpoint I want to use is 'infer'

mgoin commented 1 month ago

Hey @Alex-SSK here is a simple example of how to hit the yolo entrypoint with an image and get bounding boxes back.

import requests
import json

url = 'http://0.0.0.0:5543/v2/models/yolo/infer/from_files'
path = ['lion.jpeg'] # list of images for inference
files = [('request', open(img, 'rb')) for img in path]
resp = requests.post(url=url, files=files)
annotations = json.loads(resp.text) # dictionary of annotation results
bounding_boxes = annotations["boxes"]
labels = annotations["labels"]
print(labels)

If you need a whole example for annotating an image with the boxes, I can share that later, just let me know!