IPEX Fusions for BF16

[sreelekshmyanil]

I tried running ipex.optimize followed by tracing/scripting. I am not able to see any fusion groups in IR (torch.jit.last_executed_optimized_graph()). Is there any way to get the fusion groups other than through verbose?

[jgong5]

Do you mean by "fusion group"? We rewrite the graph by replace the matched patterns to fused ops. What do you expect to see?

[sreelekshmyanil]

So, when we use onednn graph api (torch.jit.enable_onednnfusion(True), in the IR, we can observe fusion groups as OneDNNFusionGroup . Does ipex also do graph fusions in the same way? I read in the documentation that, graph optimizations are enabled in ipex by default( ipex.enable_onednn_fusion). I want to know whether both are same.

Also, is there any other way to see fused ops, other than through verbose post-ops?

[sanchitintel]

we can observe fusion groups as OneDNNFusionGroup_

With Intel Extension for PyTorch, you'll see a slightly different name for oneDNN Graph fusion groups, but only with int8 static quantization (you could also use BF16 with AMP alongside int8 static quantization, and only with a processor that supports the AVX512_BF16 ISA).

Does ipex also do graph fusions in the same way? I read in the documentation that, graph optimizations are enabled in ipex by default( ipex.enable_onednn_fusion). I want to know whether both are same

Intel Extension for PyTorch also supports fusions that don't use oneDNN Graph, and some also use oneDNN primitives + post-ops instead of oneDNN Graph. For BFloat16, oneDNN Graph is not officially supported by Intel Extension for PyTorch (but you could still get good performance with BFloat16 while using Intel Extension for PyTorch).

[vishnumadhu365]

@sreelekshmyanil was your query answered ?

[YuningQiu]

Close this issue for now. Please feel free to re-open this issue or create a new one if you have an further questions or concerns. Thanks a lot!