I am currently using the Qwen2-VL model for an image tagging task, where I need to process all images within a folder. However, I encountered limitations in processing speed and resource utilization, and I would like to optimize this by leveraging multi-GPU parallel inference and batch processing. Here is the detailed requirement:
Multi-GPU Parallel Processing: I want to use multiple GPUs concurrently to speed up the inference process across the images in the folder.
Batch Processing: Instead of using a for loop to handle each image individually, I prefer to batch multiple images and process them together, maximizing GPU efficiency.
Expected Outcome
Ability to perform inference across multiple GPUs in parallel.
Efficient batch processing for a folder of images without looping through individual files sequentially.
Any Potential Solutions?
If there are any recommendations, guidance on configuring multi-GPU inference, or batch processing methods, that would be highly appreciated. Please advise on the necessary adjustments or any code examples to achieve this.
I am currently using the Qwen2-VL model for an image tagging task, where I need to process all images within a folder. However, I encountered limitations in processing speed and resource utilization, and I would like to optimize this by leveraging multi-GPU parallel inference and batch processing. Here is the detailed requirement:
forloop to handle each image individually, I prefer to batch multiple images and process them together, maximizing GPU efficiency.Expected Outcome
Any Potential Solutions?
If there are any recommendations, guidance on configuring multi-GPU inference, or batch processing methods, that would be highly appreciated. Please advise on the necessary adjustments or any code examples to achieve this.
Thank you for your assistance!