Open nlper01 opened 1 year ago
self.Cr = nn.Linear(config.cr*2, config.label_num*config.label_num) self.Lr_e1_rev = nn.Linear(config.label_num*config.label_num, config.lstm_hid_size) self.rounds = config.rounds # TREM 迭代次数 self.e_layer = DecoderLayer(bert_config) # TREM 中的多头注意力机制 self.label_num = config.label_num torch.nn.init.orthogonal_(self.Cr.weight, gain=1) # 正交初始化 torch.nn.init.orthogonal_(self.Lr_e1_rev.weight, gain=1) self.Lr_e1 = nn.Linear(config.lstm_hid_size, config.lstm_hid_size) torch.nn.init.orthogonal_(self.Lr_e1.weight, gain=1) self.Lr_e2 = nn.Linear(bert_config.hidden_size,bert_config.hidden_size) torch.nn.init.orthogonal_(self.Lr_e2.weight, gain=1) self.Lr_e2_rev=nn.Linear(config.label_num*config.label_num, config.lstm_hid_size)
你好,想问一下以下变量是代表什么含义,没看明白 config中的cr Lr_e1_rev /Lr_e2_rev Lr_e1/Lr_e2
以及为何要用 torch.nn.init.orthogonal_(self.Cr.weight, gain=1) # 正交初始化权重
你好,想问一下以下变量是代表什么含义,没看明白 config中的cr Lr_e1_rev /Lr_e2_rev Lr_e1/Lr_e2
以及为何要用 torch.nn.init.orthogonal_(self.Cr.weight, gain=1) # 正交初始化权重