This PoC adds a backend to lm-eval so that it can call a running TGIS or tgis-vllm server over grpc. It can run benchmarks based on the generate function for decoder and encoder-decoder models. For the logprobs function only decoder models are supported because tgis doesn't return the input logprobs for encoder-decoder models.
(Moving this PR over here)
This PoC adds a backend to lm-eval so that it can call a running TGIS or tgis-vllm server over grpc. It can run benchmarks based on the generate function for decoder and encoder-decoder models. For the logprobs function only decoder models are supported because tgis doesn't return the input logprobs for encoder-decoder models.