-
I'm getting this error when trying to use the TextGeneration node. I think it has to do with this line in TextGeneration.py
config_list = LLM['LLM'],
not adhering to the correct format required by h…
-
### What happened + What you expected to happen
Our OSS library launches Ray tasks on a cluster on behalf of our users. One user (@dcavadia) reported (https://github.com/run-house/runhouse/issues/9#i…
-
### Your current environment
```text
The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12…
-
### 报错描述
使用swift infer命令,do_sample=True时报错,do_sample=False时可以推理但生成结果乱码
### 环境
- NPU:昇腾910B3
- Python:3.9.18
- ms-swift:2.4.0.post1
- torch-npu:2.1.0
- Transformers:4.37.2
### 推理模型
Qwen2-7…
-
Hi,
Is there a way to use GGUF with BIGDL? For example, I would like to run llama-2-13b-chat.Q4_0.gguf.
Thx...sanjay
-
## [LangChain Development](https://app.pluralsight.com/library/courses/langchain-development/table-of-contents)
by [Tom Taulli](https://app.pluralsight.com/profile/author/tom-taulli)
founder : H…
-
**Describe the bug**
`cortex run mistral:tensorrt-llm` shows loads model, but running chat returns 500 error
**To Reproduce**
1. cortex run mistral:tensorrt-llm
2. cortex run mistral:tensorrt-ll…
-
I tried the inference by using BigDL LLM optimized Qwen-14B-Chat model in Aliyun (64 core, Intel(R) Xeon(R) Platinum 8369B, 256GB). For simple test case (one simeple question), there was a big perfoma…
-
### What are you trying to do?
I want to know if my computer can support the model or not, but currently no one can tell me.
### How should we solve this?
Add the memory needed for each model tag i…
-
bash scripts/webui.sh
Traceback (most recent call last):
File "/home/machinelawing/LLM/LaWGPT/utils/callbacks.py", line 47, in gentask
ret = self.mfunc(callback=_callback, **self.kwargs)
…