-
### **I am trying to Deploy and inference the XLM_Roberta model on TRT-LLM.**
I followed the example guide for BERT and built the engine: (https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/be…
-
Llama.cpp has updated the code `if (arg == "--lora") {
CHECK_ARG
params.lora_adapter.emplace_back(argv[i], 1.0f);
return true;
}` in common/common.cpp to `if (arg == "-…
-
I have set up roaming to packetbroker for ttn.
My configuration is:
```toml
# Backend Interfaces configuration (optional).
[backend_interfaces]
# interface:port to bind the Backend Interfac…
-
This only matters when the same sd_ctx is used for multiple prompts - Loras that have been applied in a previous prompt but don't appear in the current prompt are not unapplied.
Steps to reproduce:…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
2024-11-04 10:14:53.454810: E external/local_xla/xla/stream_executor/cuda/cud…
-
once i get "you don't have state dict", i can't generate an image with the sd model that is set, even if i complete the state dict, due to "'NoneType' object has no attribute 'sd_checkpoint_info'"
on…
-
### Your current environment
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.5 LTS (x86_64)
GCC ve…
-
### Expected Behavior
Lora should load with minimal vram overhead (considering it is a small lora; the 4/4 rank one is 40mb).
### Actual Behavior
Large vram usage increase when loading certain lora…
-
After placement of the Lora Loader I've got an error:
```
Error handling request
Traceback (most recent call last):
File "G:\SD\Packages\ComfyUI\venv\lib\site-packages\aiohttp\web_protocol.py"…
-
Hello,
I intend to send and receive data using Class C but except for join request acceptance nothing is happening.
No payload is sent and received. serial monitor says only "joining", nothing else.…