Can we print a model summary of CodeT5 model?

[leemgs]

I want to know how to print a model summary of CodeT5-base model that are archived at the huggingface webpage.

• https://huggingface.co/Salesforce/codet5-base
• https://arxiv.org/abs/2109.00859

We could print the model summary with the torchinfo and torchsummary modules as follows in the case of AlextNet. In the case of CodeT5-base model, How can we print the model summary?

• Case: Alexnet

  from torchsummary import summary
  help(summary)
  import torchvision.models as models
  alexnet = models.alexnet(pretrained=False)
  alexnet.cuda()
  summary(alexnet, (3, 224, 224))
  print(alexnet)

• Case: Convnet

  from torchinfo import summary
  model = ConvNet()
  batch_size = 16
  summary(model, input_size=(batch_size, 1, 28, 28))

• Case: BERT

  import torch
  from torchvision import models
  from torchsummary import summary
  dt = 2020
  torch.manual_seed(dt)
  torch.backends.cudnn.deterministic = True
  from transformers import BertTokenizer
  pretrainedmodel_vgg = models.vgg16()
  BT = BertTokenizer.from_pretrained('bert-base-uncased')
  len(BT)
  bertresult = BT.tokenize('Hi!! Welcome To The PythonGuides')
  print(bertresult)
  summary(pretrainedmodel_vgg, (3, 224, 224)

• Case Lightening (encoder/decoder):

  
  import os
  import torch
  from torch import nn
  import torch.nn.functional as func
  from torchvision.datasets import MNIST
  from torch.utils.data import DataLoader, random_split
  from torchvision import transforms
  import pytorch_lightning as pylig
  class litauto_encoder(pylig.LightningModule):
  def __init__(self):
      super().__init__()
      self.encoder = nn.Sequential(nn.Linear(28 * 28, 128), nn.ReLU(), nn.Linear(128, 3))
      self.decoder = nn.Sequential(nn.Linear(3, 128), nn.ReLU(), nn.Linear(128, 28 * 28))
  
  def forward(self, m):
      embding = self.encoder(m)
      return embding
  
  def training_step(self, btch, btch_indx):
      m, n = btch
      m = m.view(m.size(0), -1)
      o = self.encoder(m)
      m_hat = self.decoder(o)
      losses = func.mse_loss(m_hat, m)
      self.log("train_loss", losses)
      return losses
  
  def configure_optimizers(self):
      optim = torch.optim.Adam(self.parameters(), lr=1e-3)
      return optim
  dt = MNIST(os.getcwd(), download=True, transform=transforms.ToTensor())
  trained, valid = random_split(dt, [55000, 5000])

autoencoder = litauto_encoder() traine = pylig.Trainer() traine.fit(autoencoder, DataLoader(trained), DataLoader(valid)) summary(litauto_encoder,(1,28,28))

[yuewang-cuhk]

Hi, we did not experiment with torchsummary. To know more details of CodeT5 model, you can directly print the model and get the model size using this function.

[leemgs]

Hi, we did not experiment with torchsummary. To know more details of CodeT5 model, you can directly print the model and get the model size using this function.

Thanks. 👍