-
Good day
After load saved lora model, i save it to merged. And after load it from merged, i have generation like '+++++ 1000000000000000000000000000000000000000000000000…
-
### System Info
TypeScript 5.5.4
transformers.js 3.0.2
Node.js v20.170
### Environment/Platform
- [X] Website/web-app
- [ ] Browser extension
- [X] Server-side (e.g., Node.js, Deno, Bun)
- [ ] De…
-
Dear @shewu-quic, @cccclai, ..please mention anyone relevant
Could you share the command used to create the R matrix for generating the Llama-3.2-3B & 1B SpinQuant-INT4-E08 .pth files you've releas…
-
Use case: Users have pre-provisioned PVs that contain models on them and support ReadManyOnly. The user would be responsible for ensuring a compatible model is stored on the PV and creating a PVC.
…
-
### System Info
Collecting environment information...
PyTorch version: 2.2.2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 13.6.6 (x86_64)
G…
-
We want to deploy https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit which is 4-bit quantized version of llama-3.2-1B model. It is quantized using bitsandbytes. Can we deploy this using ten…
-
We want to deploy https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit which is 4-bit quantized version of llama-3.2-1B model. It is quantized using bitsandbytes. Can we deploy this using ten…
-
### Background Description
Ref: https://github.com/ggerganov/llama.cpp/pull/7553 , required for supporting future vision models (https://github.com/ggerganov/llama.cpp/issues/8010)
I initially pla…
-
To get this to work, first you have to get an external AMD GPU working on Pi OS. The most up-to-date instructions are currently on my website: [Get an AMD Radeon 6000/7000-series GPU running on Pi 5](…
-
### System Info
Python : 3.12.4
pandasai : 2.2.14
ibm_watsonx_ai : 0.2.6
### 🐛 Describe the bug
from pandasai import SmartDataframe
import pandas as pd
from pandasai.llm import IBMwatsonx
#…