Open vedaanta opened 2 months ago
CuDNN can support RN50 batchnorm fusions (and their symmetric backward counterparts) like:
Above fusions also support fp8 quantization around them.
RN50 benchmark that is being added in #443, can benefit from cudnn fusions at:
# BN-relu fusion self.bn1 = norm_layer(planes) self.relu = nn.ReLU(inplace=True) # BN-Add-relu fusion out = self.bn3(out) ... out = out + identity out = self.relu(out)
Add support for BN fusions via cudnn executor.
CC @IvanYashchuk @kshitij12345 @Anerudhan
We now have resnet50 from TorchVision natively, and #451 will add a benchmark.
🚀 Feature
CuDNN can support RN50 batchnorm fusions (and their symmetric backward counterparts) like:
Above fusions also support fp8 quantization around them.
Motivation
RN50 benchmark that is being added in #443, can benefit from cudnn fusions at:
Pitch
Add support for BN fusions via cudnn executor.
CC @IvanYashchuk @kshitij12345 @Anerudhan