-
I was wondering how you make an API request to use the custom model from the network volume. It seems that if we use
```
{"input": ..., "4":{
"inputs":{
"ckpt_name":"realistic_vision_v5.1.…
-
Hi, I am using a cloud solution (runpod) to run Forge WebUI. The Forge WebUI version that I am using dates back to the 28th of July 2024 due to the docker image that I am using. Unfortunately, there i…
-
In folders - where do we need to upload the LORA files ? I want to add multiple LORAs - Can someone clearly define the path to the LORA adding folder ?
I went in extentions and added there but it …
-
i'm enabling gpu-acceleration during installation as suggested in readme:
`CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1
pip install 'llama-cpp-python[server]'`
Then activating the local server by:…
-
Thanks for working on this!
I've been testing running embeddings in a runpod serverless environment, but the performance isn't what I would have expected. For running bge-m3, we're seeing an end to…
-
ComfyUI provides a WebSocket-API that can be used to get notified when the workflow is completed. We should use that instead of polling, because it might be way faster.
-
There are multiple open PRs looking to add inference adapters for services that offer OpenAI-compatible APIs.
* databricks - https://github.com/meta-llama/llama-stack/pull/83
* sambanova - https:/…
-
Hi
This provisioning script works fine but I noticed a certain node always fails to install with the script.
If I use the `default.sh` and add the node `"https://github.com/Gourieff/comfyui-react…
-
Really need to get GPU compute off of local hardware. RunPod seems like good option, the only rub is we need access to the encoding and the logits from the model. Also, our perplexity ratio KL diverg…
-
When using the worker with the image `runpod/worker-infinity-embedding:stable-cuda12.1.0`, with this env var `MODEL_NAMES`: `jinaai/jina-embeddings-v2-base-de`, we see this error:
```
The transfor…