vllm does not remove VRAM cache at multi-gpu environment

Marker-Inc-Korea / AutoRAG

RAG AutoML Tool - Find optimal RAG pipeline for your own data.

Apache License 2.0

1.24k stars 106 forks source link

Open vkehfdl1 opened 1 month ago

vkehfdl1 commented 1 month ago

This is the zip file that happened this error.

vkehfdl1 commented 1 month ago

We don't have multi-gpu environment, so this bug is hard to reproduce.