-
After downloading the 405B Model, I try to run the tool convert_llama_ckpt.py but I keep getting this error. I am using :
Compute: n2d-highmem-96 with 768 GB of memory on Vertex Workbench
Python ver…
-
### 🚀 The feature, motivation and pitch
Create the distribution of AMD ROCm GPU like the distributions/meta-reference-gpu which is base on NVIDIA GPU.
### Alternatives
_No response_
### Additional…
-
I have setup ipex-llm by following [install ipex-llm for llamacpp]( https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md#1-install-ipex-llm-for-llamacpp…
-
hi please see below,
I can download these directly no problem, but when I try though LMStudion this fails.
I'm on win10, been using your amazing software for a few good months now, this is happeni…
-
We did following :
1. Took nvidia/Llama-3.1-Nemotron-70B-Instruct-HF base model and performed fine tuning using our custom data set for classification task . Training completed in 6 hrs or so and w…
-
## Describe the bug
LLAMA 3.2 11B Vision cannot start after loading model
```
Error: DriverError(CUDA_ERROR_INVALID_PTX, "a PTX JIT compilation failed") when loading utanh_bf16
```
my…
-
-
I encountered an issue when trying to export a GGUF model file for Mistral Nemo and Mistral 7B finetunes using the `unsloth` library. The error occurs during the `save_pretrained_gguf` function call, …
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
### Proposal to improve performance
_No response_
### Report of performance regression
Model: meta-llama/Meta-Llama-3-8B-Instruct
GPU: 1x A6000
| SamplingParams.logprobs | Generation Throughput…