search

ChunyuanLI / Optimus

Optimus: the first large-scale pre-trained VAE language model
368 stars 37 forks source link

One question about the decoder of vae #17

Open Hsintien-Ng opened 3 years ago

Hsintien-Ng commented 3 years ago

File: https://github.com/ChunyuanLI/Optimus/blob/master/code/examples/big_ae/modules/vae.py

code in line 188, 133, 143: outputs = self.decoder(input_ids=labels, past=latent_z, labels=labels, label_ignore=self.pad_token_id) this line takes labels as the input_ids of decoder. I wonder know if it is an error. Should it be input_ids=inputs? Since there exits -1 in labels, it occurs an error in line 460 (inputs_embeds = self.wte(input_ids)) in modeling_gpt2.py in pytorch_transformers.

Thanks.