Closed rahul-tuli closed 6 months ago
This PR fixes a few bugs found while updating the eval framework:
1) sequence_length is no longer exposed at the pipeline level
sequence_length
from deepsparse import Pipeline pipeline = Pipeline.create( task="text-generation", model_path="zoo:mpt-7b-mpt_pretrain-base_quantized", ) assert hasattr(pipeline, "sequence_length")
Traceback (most recent call last): File "/home/rahul/projects/deepsparse/local.py", line 12, in <module> assert hasattr(pipeline, "sequence_length") AssertionError >>>
Fix: workaround relies on pipeline.ops["single_engine"].sequence_length
pipeline.ops["single_engine"].sequence_length
2) deepsparse.eval requires a specific commit from lm-eval-harness for now, updating the install message to reflect that
deepsparse.eval
This PR fixes a few bugs found while updating the eval framework:
1)
sequence_length
is no longer exposed at the pipeline levelFix: workaround relies on
pipeline.ops["single_engine"].sequence_length
2)
deepsparse.eval
requires a specific commit from lm-eval-harness for now, updating the install message to reflect that