-
Hi, I would like to try meta-llama/Llama-3.2-1B in different scenario of IPEX-LLM solutions.
pip list:
bigdl-core-xe-21 2.6.0b20241001
intel-extension-for-pytorch 2.1.10+xpu
intel-ope…
-
### The model to consider.
https://huggingface.co/SparseLLM/prosparse-llama-2-7b
### The closest model vllm already supports.
Llama
### What's your difficulty of supporting the model you want?
So…
-
edit: both 3.1 and 3.2 fail
- 3.1 support is fixed by `pip install transformers==4.43.2` - thank you, @jinxiangshi
- 3.2 isn't yet supported by TRT-LLM - @laikhtewari promised to update the docs to …
-
decoder_base: {path to store the official weights of LLaMA-2}
which model is the official weights of LLaMA-2
-
### Your current environment
```text
The output of `python collect_env.py`
```
```
:128: RuntimeWarning: 'torch.utils.collect_env' found in sys.modules after import of package 'torch.utils', bu…
-
## 🐛 Bug
## To Reproduce
Steps to reproduce the behavior:
I followed [https://captum.ai/tutorials/Llama2_LLM_Attribution](url)
My code is here，the only difference is I changed the model_…
-
It's trying load and never completed
```
Removing download task for Shard(model_id='llama-3.2-1b', start_layer=0, end_layer=15, n_layers=16): True
0%| …
-
Is it working right now in any way?
-
### Description
Using structured output with vision models like gpt-4o-mini works. I'd like to do the same for Llama-3.2-11B-Vision-Instruct from GitHub models. Currently it throws an exception.
##…
-
### System Info
CPU: x86_64, memory: 1024GB, GPU: 8*A6000 48GB each, Tensorrt-LLM version 0.9.0.DEV20240226. NVIDIA-Driver Version: 535.171.04 CUDA Version: 12.2; OS - Ubuntu 22.04
### Who can hel…