Example `gpu_device_ids` for multi-model usage?

[ghost]

System Info

P4D (A100 40 GB x 8)

Who can help?

@juney-nvidia @byshiue

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

I want to use multi-model TP-4 x 2 setup. How do I set gpu_device_ids to [[0,1,2,3],[4,5,6,7]] Could you please provide an example usage of gpu_device_ids?

Expected behavior

n/a

actual behavior

n/a

additional notes

n/a

[ghost]

How do I use an ensemble which load balances across these 2 model instances?