Open gangooteli opened 8 months ago
hi! I was having the same issue and was able to solve it by running
apt-get install build-essential
Hi there,
I get that installing build-essential is necessary to use CC Compiler with Triton JIT. But I'm curious as to why the compilation can't be done during the Docker build or installation of the vLLM stage instead of at runtime? This question arises due to security concerns, as the compiler in the container increases vulnerability risk.
Hi there,
I get that installing build-essential is necessary to use CC Compiler with Triton JIT. But I'm curious as to why the compilation can't be done during the Docker build or installation of the vLLM stage instead of at runtime? This question arises due to security concerns, as the compiler in the container increases vulnerability risk.
@JasmondL is there any way to solve this? Because I only build once in Dockerfile.
Hi @fahadh4ilyas ,
I'm currently unfamiliar with the topic. I'll investigate further when I have a chance.
Below is the error I am getting while using generate api with below params. First time it is able to generate with prefix_pos but next call I am getting below error.
"use_beam_search": false, "n": 1, "temperature": 0.0, "max_tokens" : 200, "skip_special_tokens" : true, "prefix_pos" : 504