search

zhengzangw / Sequence-Scheduling

PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
76 stars 15 forks source link

The meaning of different strategies #1

Closed qxpBlog closed 7 months ago

qxpBlog commented 7 months ago

@zhengzangw Sorry, can you explain the meanings of the following different strategies: seqsch、vanilla、gt、po? image

zhengzangw commented 7 months ago

Sure. seqsch is our method. vanilla is to inference without our method. gt means using our method but with a ground truth knowledge of the text length, which serves as an upper bound of our method. These can also be found in our paper.

qxpBlog commented 7 months ago

Sure. is our method. is to inference without our method. means using our method but with a ground truth knowledge of the text length, which serves as an upper bound of our method. These can also be found in our paper.seqsch``vanilla``gt

Thank you for your answer，I got it.