-
Phi-3-mini-128k-instruct has the same number of parameters and same architecture as Phi-3-mini-4k-instruct, unless I am mistaken. Would it be possible for unsloth to support inference for this model a…
-
Not the most powerful, but a useful model:
https://huggingface.co/microsoft/Phi-3-mini-128k-instruct
-
Hello!
First of all, I'm impressed by this project and I hope it will pick up some steam in the nearby future. But I would really like to see additional LLMs/SLMs, as local models get faster and mor…
-
### System Info
GPU: RTX4090
Run 2.1.0 with docker like:
`docker run -it --rm --gpus all --ipc=host -p 8080:80 -v /home/jp/.cache/data:/data ghcr.io/huggingface/text-generation-inference:2.1.0 …
-
### System Info
g6.12xlarge, docker container: ghcr.io/huggingface/text-generation-inference:sha-90184df
### Information
- [X] Docker
- [ ] The CLI directly
### Tasks
- [X] A…
-
I have downloaded the `microsoft/phi-3-mini-128k-instruct` model from huggingface using the `huggingface-cli` script.
When I try to convert the model to ONNX format, the directory specified with th…
-
### Your current environment
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (U…
-
Seems like a good match for WebLLM, as it was practicaly designed to run in the browser.
From this reddit thread:
https://www.reddit.com/r/LocalLLaMA/comments/1d2o445/comment/l63cvxk/
-
Hi. Is it possible to add version 128k of phi-3 in ollama?Thanks in advance.
-
## 🚀 Feature
[Feature Request] phi-3 small released -> performs two times better then Phi-3 mini
https://huggingface.co/microsoft/Phi-3-small-128k-instruct
## Motivation
Phi-3 **small** just…