-
I followed installation guide https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Overview/install_gpu.html
in order to run IPEX-LLM on iGPU of Meteor Lake with Windows OS platform.
**---Steps to …
-
**LocalAI version:**
`local-ai` 2.1.0 (TrueCharts chart Version: 6.6.1)
Cublas Cuda 11 + FFmpeg image
**Environment, CPU architecture, OS, and Version:**
uname -a
```
Linux truenas 6.1.63-prod…
-
log:
```
root@bm1684:/data/LLM-TPU/models/Llama3/python_demo# python3 pipeline.py -m /data/models/llama3-8b_int8_1dev_512.bmodel -t ../token_config/ --devid 0
None of PyTorch, TensorFlow >= 2.0, or…
-
### Describe the bug
MODEL_ID="/models/models--EleutherAI--gpt-neox-20b"
mkdir saved_results_gpt_neox
python run_gpt-neox_int8.py --ipex-weight-only-quantization --output-dir "saved_results_gpt_neo…
-
v1 of Dot (download), macOS 14.4 (23E214). This occurs in both small and big models.
![CleanShot 2024-03-17 at 11 30 55@2x](https://github.com/alexpinel/Dot/assets/980297/faa62ccf-65a7-4cbc-8203-a…
-
Hi,
I'm trying to play with localGPT to understand how LLM are working, within a lxc container on my proxmox server (it's a homelab based on i5-1340P 16vpu/32go ram).
I try several engines to c…
-
I want to use nsight system to profile vllm.
Here is my code:
Seems by default, Ray is not used, is that right?
But my nsys profile still fails. Why is that?
Thank you.
```python
def ini…
-
What is a correct way to build edgen locally in Mac with metal?
```bash
git clone https://github.com/edgenai/edgen.git
cd edgen/edgen
npm run tauri build
```
This seems to always crash with …
-
Hi,
I have ported the Alpaca Qlora code given for GPU example, to CPU. I am using Sapphire Rapids for training.
These are my code changes:
model = AutoModelForCausalLM.from_pretrained( …
-
### Your current environment
My current environment setup is :
```
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.1.2
[pip3] triton==2.1.0
[conda] numpy …