How to run YOLOv8x across multiple GPUs?

[onefanwu]

Search before asking

• [X] I have searched the EvaDB issues and found no similar questions.

Question

When I set ray to True and gpu_ids to '[0,1,2,3,4,5,6,7]', YOLOv8x is only running on cuda:0 and not using other GPUs. Did I set it up wrong?

I use the EvaDB v0.3.8, and the used queries are as follows:

SET ray=True;
SET gpu_ids='[0,1,2,3,4,5,6,7]';
SET batch_mem_size=300000;

DROP TABLE IF EXISTS NightStreet;

LOAD VIDEO 'benchmark/datasets/video/night-street.mp4'
INTO NightStreet;

DROP TABLE IF EXISTS Yolo;

CREATE FUNCTION IF NOT EXISTS Yolo
TYPE  ultralytics
MODEL 'benchmark/models/yolov8x.pt';

SELECT id, Yolo(data).labels
FROM NightStreet
WHERE ['person', 'car'] <@ Yolo(data).labels;

[onefanwu]

When I modified execution_context.py, the parsing of gpu_ids became normal. I don't know if this is the case.

    def _populate_gpu_from_config(self) -> List:
        available_gpus = [i for i in range(get_gpu_count())]
        user_gpus = []
        if isinstance(self._user_provided_gpu_conf, str):
            user_gpus = json.loads(self._user_provided_gpu_conf)
        return list(set(available_gpus) & set(user_gpus))

But still only cuda:0 is used, is there a way to get 8 GPUs to compute YOLO.

[xzdandy]

You need ray to run it across multiple GPUs. Is the issue you mentioned in #1357 fixed?

[onefanwu]

Hi @xzdandy , Is there a tutorial on getting Ray to run on multiple GPUs in EvaDB? I have fixed this issue ,https://github.com/georgia-tech-db/evadb/issues/1357 .

[onefanwu]

When I set CUDA_VISIBLE_DEVICES, Ray seems to work fine.

[xzdandy]

Add instructions about seting CUDA_VISIBLE_DEVICES in https://evadb.readthedocs.io/en/stable/source/overview/faq.html