Open yorickvP opened 3 months ago
Breaking changes:
enable_trt_overlap
max_queue_size
max_seq_len
Other changes:
Breaking changes:
enable_trt_overlap
now uselessmax_queue_size
is now required, 0 is a good defaultmax_seq_len
required until next week, but will default to the model max input size afterOther changes: