search

microsoft / antares

Antares: an automatic engine for multi-platform kernel generation and optimization. Supporting CPU, CUDA, ROCm, DirectX12, GraphCore, SYCL for CPU/GPU, OpenCL for AMD/NVIDIA, Android CPU/GPU backends.
Other
435 stars 45 forks source link

Is there any document for performance benchmark result vs pytorch2.1 compile mode? #377

Open fsword73 opened 3 months ago

fsword73 commented 3 months ago

Pytorch 2.1 compile mode supports fused kernels which updated the performance flag.
Is there any document to compare the performance with pytorch 2.1 compile mode . For example, 5 CNN models and 5 LLM models. It is better to have per layer compare.
In this way , developers can decide to move this software stack or not. Usually developers does not expect to benchmark before baseline data were available

ghostplant commented 2 months ago

Hi @fsword73. Thanks for your questions.

First, Pytorch 2.1 compile mode & Antares don't have any conflicts between each other which allows users to enable both to maximize everything, so I'm confused of the context to "move this software stack".

For benchmarking, your suggestion is great, can you share some concrete model repos that are optimized by Pytorch 2.1 compile mode? We didn't keep track on Pytorch 2.1 compile mode in the past. To avoid any unfairness of comparison, we'll use your suggested repo to do benchmarking.