-
Hi, I'm fine tuning an LLM using Soft Prompt Tuning using DeepSpeed via Accelerate implicitly, using the `deepspeed` param in `TrainingArguments`.
And all goes well until after the first epoch, whe…
-
### Describe the issue
We are currently developing a system that involves deploying Large Language Models (LLMs) on Android smartphones. To date, we've managed to execute inference tasks using ONNX R…
-
python = 3.11.5
torch = 2.1.0 + cu121
vllm = 0.2.2
GPU: L40 * 4
I install vllm by "pip install vllm".
It will STUCK when loading vicuna-7b-v1.5 model using the vllm framework, while the fastc…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
### Describe the bug
MODEL_ID="/models/models--EleutherAI--gpt-j-6b"
python run_gpt-j_int8.py -m ${MODEL_ID} --quantized-model-path "./saved_results_gptj/best_model.pt" --benchmark --jit \
--token-…
-
Trying to deploy Mixtral-8x7B-Instruct-v0.1 using notebook https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_mistral.ipynb.
Mo…
jvovk updated
7 months ago
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
### Your current environment
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (U…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
**Describe the bug**
When I ran mpt model on the CPU, I encountered the following error.
![image](https://github.com/microsoft/DeepSpeed/assets/97155466/440d34f3-7881-4cba-aa8d-22d1c3207b9b)
**To…