-
Thank you great work!
I have a few questiones about peft? I hope you can answer that. Thank you a lot!
1. Which model fine-tuning is best to use? Is it a pre-trained model (llama2-7b) or a supervise…
zsxzs updated
1 month ago
-
### Proposal to improve performance
_No response_
### Report of performance regression
_No response_
### Misc discussion on performance
---
**Setup Summary for vLLM Benchmarking with Llama…
-
您好，我在使用您提供的代码中/reproduce/utils.py中的merge_llama函数和/examples/merge.py，对用meta-math按照论文参数设置训练的llama2-7b进行merge后，模型总大小从原先的12G扩大到了20G，并且无论输入总输出大段空白，但对t5-base进行merge前后模型大小几乎相等，想问下这是正常的吗？图中是merge函数。
训练的命令：
…
-
Hi SolidGPT team,
Are there any ways I can deploy your product with Llama2 or any other open source model?
I saw you have [Llama manager](https://github.com/AI-Citizen/SolidGPT/blob/main/solidg…
-
### 🐛 Describe the bug
when I prepare a llama2 exe file as https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md#step-4-run-on-your-computer-to-validate description, however…
-
hi! I am a undergraduate student who interested in your team's project. When I run demo code in EE-Tuning part, I found there are no tuning scripts for llama2-7b model (truly provided for 13b and 70b)…
-
### Describe the bug
I have been able to create my local bolt.new and use API keys to connect with the Ai models but the biggest issue is with Ollama and getting this set up. I have been using curs…
-
I use this command to quantize llama2-7b-chat model, but the model size dosen't change.
CUDA_VISIBLE_DEVICES=0 python3 main.py \
--model /mnt/home/model/llama2-7b-chat-hf \
--epochs 20 --o…
-
环境：操作系统win10，python版本v3.10.14，llama_cpp_python库的版本v3.1.0
直接跑项目里的自带的llama1.py或llama2.py文件都会报错（gguf模型文件已下载且路径正确）
比如执行python llama2.py后的错误信息如下：
![image](https://github.com/user-attachments/assets/72d7…
llery updated
1 month ago
-
dequantizelinear_kernel on the llama2 7B 16a4w model is not fusing as expected