It looks like there is a code issue somewhere as the variables for EMBED_NIM_GPUs and RERANKER_NIM_GPUS seem not to work. Although I put in device=1 to pin the GPU to my GPU1 it tries to run on all GPUs. If I hardcode device=1 in the start scripts of this project it works as expected. Strangely, the variable seems to work for the LLM_NIM as I put in device=0 here and it pins to just GPU0
It looks like there is a code issue somewhere as the variables for EMBED_NIM_GPUs and RERANKER_NIM_GPUS seem not to work. Although I put in device=1 to pin the GPU to my GPU1 it tries to run on all GPUs. If I hardcode device=1 in the start scripts of this project it works as expected. Strangely, the variable seems to work for the LLM_NIM as I put in device=0 here and it pins to just GPU0