Closed dsikka closed 8 months ago
create()
Pipeline.create(...)
from deepsparse.v2.pipeline import Pipeline model_path = "hf:neuralmagic/mpt-7b-chat-pruned50-quant" pipeline = Pipeline.create( task="text_generation", model_path=model_path, engine_kwargs={"engine_type": "deepsparse"}, internal_kv_cache=True, continuous_batch_sizes=[2, 4], )
Summary
create()
method such thatPipeline.create(...)
can be used to initialize a pipelineTesting