search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
411 stars 133 forks source link

I don't know the cause of this error. #102

Closed amazingkmy closed 1 year ago

amazingkmy commented 1 year ago

Description

triton server 22.12

V100

Reproduced Steps

i try to install fastertransformer_backand in triton_server:22.12 (Success)
and i convert gpt-j model using examples/pytorch/gptj/utils/huggingface_gptj_ckpt_convert.py (Success)
An error occurs when I run tritonserver.

[WARNING] gemm_config.in is not found; using default GEMM algo
terminate called after throwing an instance of 'std::runtime_error'
  what():  [FT][ERROR] CUDA runtime error: API call is not supported in the installed CUDA driver /root/build/build/_deps/repo-ft-src/src/fastertransformer/utils/allocator.h:181
byshiue commented 1 year ago

Can you share the branch you use and share the nvidia-smi info?

amazingkmy commented 1 year ago

@byshiue here

Mon Mar 13 10:16:40 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 450.102.04 Driver Version: 450.102.04 CUDA Version: 11.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... On | 00000000:18:00.0 Off | 0 | | N/A 36C P0 43W / 300W | 0MiB / 32510MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+

amazingkmy commented 1 year ago

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed_Sep_21_10:33:58_PDT_2022 Cuda compilation tools, release 11.8, V11.8.89 Build cuda_11.8.r11.8/compiler.31833905_0

byshiue commented 1 year ago

It looks like the driver is too old? Can you try newer driver?

amazingkmy commented 1 year ago

@byshiue ok i try it. What version of the driver do you recommend?

byshiue commented 1 year ago

You can try the latest one (525.85.12).

amazingkmy commented 1 year ago

@byshiue solve it. ft need cuda version higher than 11.2

amazingkmy commented 1 year ago

Thanks @byshiue