-
Using this tool can generate an awful lot of log noise. See below for a sample of output from just a few minutes of use. I was reconfiguring my webpack configuration, so the output information is corr…
-
ERROR: Could not install packages due to an OSError: [WinError 5] Access Denied: 'D:\\Stability Matrix\\Packages\\ComfyUI\\venv\\Lib\\site-packages\\onnxruntime\\capi\\onnxruntime_providers_shared.dll…
-
In plumbing through the Security Assistant Knowledge Base API integration tests (https://github.com/elastic/kibana/pull/192665), I ended up having to add support for a `modelId` override to our API's …
-
Are there docs on best practices for using vllm hosted models?
I create a model using
python -m vllm.entrypoints.openai.api_server --model model_path
and try running it as
lm_eval --model lo…
-
Currently, I'm using fastchat==0.2.36 and vllm==0.4.3 to deploy Qwen model for inference service. Here's the command for starting the service on my two servers.
server1:
`python3.9 -m fastchat.serve…
-
### What happened?
In the proxy admin UI (v1.44.23 stable), I added an invalid model by mistake*, and now I'm getting constant error messages in the logs with no way I can see to stop them.
The er…
-
deployment.yaml
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tensorrt-inference-server
labels:
app: tensorrt-inference-server
spec:
replicas: 1
selector:
mat…
-
I REALLY NEED YOUR HELP. I REALLY NEED RoseTTAFold-All-Atom. After completing the installation steps, I got an error message during runtime.I tried to change the path to an absolute path in "protein.y…
-
Traceback (most recent call last):
File "/home/lhs/project/nerf...wu/depth-fm-main/inference.py", line 113, in
main(args)
File "/home/lhs/project/nerf...wu/depth-fm-main/inference.py", lin…
-
### Feature request
It would be immensely useful to have a server-application to serve up HF-Transformer and other Hub models as a service, similar to the how `llama.cpp` bundles the `llama-server`…