neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

Other

2.97k stars 171 forks source link

[Benchmarking] Update `data_creation` to work with prompt alias in `text_generation` #1462

Closed dsikka closed 8 months ago

dsikka commented 8 months ago

Summary

text_generation uses the TextGenerationInput schema which has an alias prompt for sequences. This PR updates the data_creation function such that benchmarking text_generation works. Was previously failing due to incorrect schema creation

Testing

Testing locally using an mpt model: deepsparse.benchmark_pipeline text_generation zoo:mpt-7b-mpt_pretrain-base_quantized -x v1.json

Was previously getting this error:

pydantic.error_wrappers.ValidationError: 1 validation error for TextGenerationInput
prompt
  field required (type=value_error.missing)