-
Thank you for your outstanding work！
I would like to know how much memory the device needs to have when using GPU to perform inference "python scripts/infer.py --opts-path configs/infer/lmo.json"? …
-
Hi @danielhanchen ,
I am unable to use "unsloth/gemma-2b-bnb-4bit" via vLLM. I am getting below error while loading the model on Nvidia_T4 or NVIDIA_V100 GPU .
`engine_args = EngineArgs(model="u…
-
## 🐛 Bug
I am getting CUDA re-initialize error.
I am using below code to generate synthetic dataset on GPU. To perform distributed training I am using [official PyTorch distributed training help…
-
### Anything you want to discuss about vllm.
Updated flashinfer to v0.0.9 in the following test scripts:
- Async Engine, Inputs, Utils, Worker Test
- Tensorizer, Metrics, Tracing Test
- Basic Corr…
-
**Your hardware and system info**
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
system: ubuntu 22.04
python: 3.11
GPU: 4060ti 16G…
-
If we are using local model, we need to pass device to utilise the gpu for inference. However, in `launch_concordia_challenge_evaluation.py `
```
# Language Model setup
model = utils.language_mode…
-
(equiscore) (base) [yangchh@gnode14 EquiScore]$ python Screening.py --ngpu 1 --test --test_path ./data/sample_data/ --test_name tmp_pockets --pred_save_path ./data/test_results/EquiScore_pred_for_tmp…
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
I am trying to use GPUTree explainer, however, the explainer gives me error. See below my codes:
```
explainer = shap.explainers.GPUTree(rf_model)
shap_values = explainer(X_test)
```
The last…
-
### System Info
```Shell
- `Accelerate` version: 0.35.0.dev0
- Platform: Linux-5.10.0-32-cloud-amd64-x86_64-with-glibc2.31
- `accelerate` bash location: /opt/conda/envs/flux_cn_exp/bin/accelerat…