-
qwen.py", line 303, in load_weights
param = state_dict[name]
KeyError: 'transformer.h.0.attn.c_attn.g_idx'
-
This is mostly to keep track of this problem which has been around for a while
if you ever do something like 1)quantize cpu model with int4, 2)move it to cuda
then the output of the model will be …
-
Is there a plan to support arm? I have a gh200 and would like to use EETQ for quantization. Bitsandbytes supports Int4 but int8 on GH200 is not available
-
**Describe the bug**
When building a model,
`--extra_options use_qdq=1` and `--extra_options use_qdq=0` yield the same binary model.onnx (compared using `diff`)
which differs from the `model.onnx` wh…
aendk updated
2 weeks ago
-
The app crashes when using modelPath after downloading the file from network.
```
const llmInference = useLlmInference({
storageType: 'file',
modelPath: '/data/user/0/com.offlinellmpoc/fi…
-
### Description
It works correctly for the second table, but not the first.
use code:
```
drop table if exists hello1;
drop table if exists hello2;
CREATE TABLE hello1 (
"Hello" int4 …
-
### System Info
Windows 11
Python 3.12.7 (and 3.12.5 apparently depending on running py --version or python --version from PowerShell)
### Information
- [X] The official example scripts
…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
Machine A800, VLLM 0.5.0, PROMPT=开始, output max tokens = 2048, Temperature sets 0.7
VLLM…
-
**Build Scans:**
- [elasticsearch-periodic-platform-support #4521 / oraclelinux-8_platform-support-unix](https://gradle-enterprise.elastic.co/s/mhjo3pknmbdc2)
- [elasticsearch-pull-request #37673 / pa…
-
### Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [X] I have s…