-
**The command I execued as follow**
/app/tensorrt_llm/examples/quantization/quantize.py \
--model_dir ${model_dir} \
--dtype float16 \
--qformat int8_wo \
--kv_ca…
-
### Your current environment information
Collecting environment information...
PyTorch version: 2.1.1+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A
…
-
Used the following lines for env creation
****
conda create --name unsloth_env python=3.10
conda activate unsloth_env
conda install pytorch-cuda= pytorch cudatoolkit xformers -c pytorch -c nvi…
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm …
-
### Description
python3 main.py --text "a hamburger" --workspace trial -O --backbone grid_taichi
Get this error:
[Taichi] version 1.6.0, llvm 15.0.1, commit f1c6fbbd, win, python 3.9.13
No CUD…
-
# Overview
On 2024-04-26, I emailed Sunil about two participants who appeared to be missing consent forms in FM Pro. He checked Qualtrics, which is where the consent forms are stored, and told me tha…
-
When I run `CUDA_VISIBLE_DEVICES=0,1 torchrun --standalone --nnodes=1 --nproc-per-node=2 cli_demo_sat.py --from_pretrained cogcom-base-17b --local_tokenizer tokenizer --english --fp16 (--quant 4)`, an…
-
```
Traceback (most recent call last):
File "/usr/local/bin/paddle2onnx", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/paddle2onnx/command.py", line 142, in m…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
使用量化方式加载模型时,提示
`Failed to load cpm_kernels:[WinError 267] 目录名称无效。: 'C:\\Users\\Hengj\\AppDat…
-
i run the gradio_app.py on ubuntu with RTX4090,when i upload the image and generate the video it render error as below:
RuntimeError:CUDA out of memory.Tried to allocate 1.07 GiB (GPU 0; 23.65 GiB …