Open xrsrke opened 1 year ago
Can you assign this to me? I'd like to give this a shot
@sami-bg Check out torch.fx. We could use it to detect modules in a transformers model that can be fused and replace them with the fused version: model.transformers.blocks[0].dropout = fused_dropout
But we don't do it manually. Check out this tutorial for how to use torch.fx: https://pytorch.org/tutorials/intermediate/fx_conv_bn_fuser.html
Fuse some popular functions and automatically replace modules in an existing 🤗 transformers model with their corresponding fusion module
APIs
TODOs
Reading (could be ignored)