-
```
deepspeed ./train_ppo.py \
--pretrain OpenLLMAI/Llama-2-7b-sft-model-ocra-500k \
--reward_pretrain OpenLLMAI/Llama-2-7b-rm-anthropic_hh-lmsys-oasst-webgpt \
--save_path ./ckpt/7b_l…
-
## Bug Description
Configuration : `llm_examples_main` branch, current torch version : 2.4, transformers==4.41.2
One of the subgraphs is receiving a `SymInt` node (s0+1) which relies on other `Sy…
-
```
import ollama from 'ollama'
const message = { role: 'user', content: 'What color of the sky?' }
const response = await ollama.chat({ model: 'llama2', messages: [message], stream: true })
for…
-
[https://github.com/pytorch/executorch/blob/main/examples/models/llama2/README.md#performance](https://github.com/pytorch/executorch/blob/main/examples/models/llama2/README.md#performance)
AFAIK, Q…
-
How can I replace the OpenAI API with Llama2?
-
I was wondering how to understand this. I would expect llama2 70b to have a lower throughput.
Is the configuration different between the table for llama2 70b and the table for llama2 7b.
-
@JLTastet @timinar Excuse me,how should i distill the Llama-2-7B model to obtain a 3.5B Llama-2 model by BabyLlama?At the same time, I want to use the local Llama-2-7B model whose path is ``` /home/Ll…
-
我根据模板自定义了一个模型,但是在注册时输入xinference register --model-type LLM --file model.json --persist报错。
自定义模型:
![1697530192627](https://github.com/xorbitsai/inference/assets/90330685/e1a31b57-33ba-45a0-bae5-395b9…
-
Wondering if you counted the ASR of the original LLM (pure LLaMA2-chat, Vicuna...) under the attack image prompt
-
Ref https://huggingface.co/papers/2310.11453