-
from bigdl.llm.transformers import AutoModelForCausalLM
import torch.nn.utils.prune as prune
model_path = r'D:\test_bigdl\model\Baichuan2-7B-Chat'
model = AutoModelForCausalLM.from_pretrained(mod…
-
请问可以提供下加载lora训练权重的推理脚本吗，谢谢
-
-
您好，您提供的数据集中，DISC-Law-SFT-Triplet 包含 `input`，`output` 和 `reference` 三个部分。在用 [LLaMA Efficient Tuning](https://github.com/hiyouga/LLaMA-Efficient-Tuning) 微调时，请问 `reference` 是如何加入训练的呢？我目前是把它作为 system 输入，…
-
Thanks for open source your model. I was following your HF webpage, https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat, and I asked following question:
> \>\>\> model.chat(tokenizer, [{"role": "…
-
I've followed the instruction
https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/docs/baichuan.md
to run Baichuan2-7b-Chat.
But for exactly the same engine, the outputs are …
-
请问有哪些措施可以运行，若不做量化的话？
-
Is multi-machine training of large models suitable for multi-node large models? Secondly, can the large model be divided into blocks and allocated to each node for training? For example: Chatglm3 larg…
-
模型：baichuan2-13B-chat
问题1：
复现代码块：
In [4]: import pyfastllm
In [5]: model = pyfastllm.create_model("baichuan2-int8.flm")
In [6]: prompt = model.make_input("", 0, "你好")
In [7]: prompt
Out[7]: '…
-
## Issue1 on xpu with python 3.10 [Fixed after releasing bigdl-core-xe and bigdl-core-xe-esimd for python 3.10]
on Arc14, I followed https://github.com/intel-analytics/BigDL/blob/main/python/llm/exa…