-
Ran `pip install -U -r requirements.txt -r requirements-dev.txt
python setup.py install` and got the following error below. (Yes I upgraded pip). Any ideas?
Error output below:
`
Collecting en…
-
### 请提出你的问题
在使用LLM时,执行命令:
```
python predictor.py \
--model_name_or_path meta-llama/Llama-2-7b-chat \
--dtype float16 \
--max_length 1024 \
--mode "dynamic" \
--inference_m…
-
Model: `TheBloke/Mistral-7B-OpenOrca-AWQ` (and any other Mistral AWQ model of them)
Cuda: `12.2`
```
WARNING 12-03 17:13:44 config.py:398] Casting torch.bfloat16 to torch.float16.
WARNING 12-03 …
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and dis…
-
Maybe have a context manager?
* Need a way to `inject` params & automatically hash the params to create instance names (to leverage the cache)
* Need a way to expose the instance names
* Integrat…
-
Feature sequence parallel has already support by Megatron and DeepSpeed for a long time.
Input samples are not only separated at batch dim, but also at sequence length dim when using tensor paralleli…
-
## How to reproduce the behaviour
I'm getting an "Unknown function registry: 'llm_backends'" error (see the traceback below) when running the example provided in Matthew Honnibal's blog "Agains…
-
Hello!
I've been working on getting MiniGPT4 working on Inf2. While I have been able to get the LLM portion running successfully on neuron, I am struggling to get to get the ViT model working.
H…
-
Otherwise I encountered the below on SPR:
```bash
cannot import name 'dispatch_model' from partially initialized module 'accelerate' (most likely due to a circular import) (/root/anaconda3/envs/heya…
-
vllm==0.1.5
```
INFO 09-08 09:55:52 llm_engine.py:72] Initializing an LLM engine with config: model='baichuan2', tokenizer='baichuan2', tokenizer_mode=auto, trust_remote_code=True, dtype=torch.bfloa…