search

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs
https://neuralmagic.com/deepsparse/
Other
2.97k stars 171 forks source link

[V2 Pipeline] SImple Asyncio pipeline test #1478

Open horheynm opened 8 months ago

horheynm commented 8 months ago

Description

Adds tests for Pipeline.run_async()

Problem

Testing run_async() currently requires some hacking in tests/server. Isolate Pipeline func's test.

Solution

Simple pipeline running run_async

Usage

    inference_state = InferenceState()
    inference_state.create_state({})
    pipeline_input = IntSchema(value=5)

    pipeline_output = await AddThreePipeline.run_async(
        pipeline_input, inference_state=inference_state
    )

Note

using asyncio-pytest works fine locally, but when Pipelines are registered during pytest collection, it has conflicts Ex. 

INTERNALERROR> RuntimeError: Attempting to register name image_classification as <class 'src.deepsparse.image_classification.pipeline.ImageClassificationPipeline'> however image_classification has already been registered as <class 'deepsparse.image_classification.pipeline.ImageClassificationPipeline'>

So created homemade solution to run asyncio