Closed larryzhang23 closed 2 years ago
Oh yea thats correct. My bad. Thanks for correcting. Although it doesnt seem to harm performance that bad, Im currently training another model at 32x32 resolution. Thats the loss curve:
I restarted training and it seems to perform better now.
(Green is the fixed tokens)
But I also changed something at the optimiter weight decay, so maybe this also has some effect on it.
Hi, In transformer.py, I find mask_token_id is set to be args.num_image_tokens. Shouldn't it be the args.num_codebook_vectors? I think we don't want the mask token id to be one of those in the codebook. Similar thing for the sos token id.