Summary

We hadn't yet added in a parse_inputs function to the new TextGenerationPipeline.
The base parse_inputs function previously in BasePipeline is covered by the normal operator functionality but for TextGeneration, we do a bit of extra kwarg/arg parsing. This operator handles that additional functionality
Testing
All integration and unit testing pass

Examples of cases which now work:

With the following pipeline:


pipeline = Pipeline.create(
    task="text_generation",
    model_path=model_path,
    engine_type="deepsparse",
    internal_kv_cache=True,
)

Not explicitly listing prompt or sequences

output = pipeline(
    "What is your favourite snack?", num_return_sequences=2, do_sample=True
)

Listing sequences:

output = pipeline(
    sequences="What is your favourite snack?", num_return_sequences=2, do_sample=True
)

Listing prompt:

output = pipeline(
    prompt="What is your favourite snack?", num_return_sequences=2, do_sample=True
)

Passing generation parameters as kwargs:

output = pipeline(
    "What is your favourite snack?",  num_return_sequences=2, do_sample=True)

Passing generation_config and overriding with kwargs:

output = pipeline(
    "What is your favourite snack?", generation_config={"max_length": 50}, output_scores=True
)

neuralmagic / deepsparse