search

tkestack / vcuda-controller

Other
488 stars 156 forks source link

CUBLAS_STATUS_ALLOC_FAILED (cuda11.4.4，cuBLAS 11.6.5.2) #33

Closed work-chausat closed 1 year ago

work-chausat commented 1 year ago

当上层调用cublasCreate的时候，程序会alloc失败，报：CUBLAS_STATUS_ALLOC_FAILE

hzliangbin commented 1 year ago

@work-chausat could you try this commit? We've encountered similar problem, and fixed it.

work-chausat commented 1 year ago

@work-chausat could you try this commit? We've encountered similar problem, and fixed it.

@hzliangbin @VincentLeeMax @mYmNeo 算力控制不住，gpu拆分失败。如下图算力设置为整卡20%，但实际利用率 top_result.user_current已到整卡99%

截屏2022-11-18 16 17 59
hzliangbin commented 1 year ago

@work-chausat how about GPU memory ？is it isolated？