Some questions about the code

[Eldo-rado]

Hi, thanks for your code! And i have some questions to confirm.

1. unmatched weight

elif args.bert == 'sci':
            self.text_decoder = BertLMHeadModel.from_pretrained('allenai/scibert_scivocab_uncased',config=decoder_config)

The error here is

RuntimeError: Error(s) in loading state_dict for BertLMHeadModel:
    size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([31090, 768]) from checkpoint, the shape in current model is torch.Size([31092, 768]).
    size mismatch for cls.predictions.bias: copying a param with shape torch.Size([31090]) from checkpoint, the shape in current model is torch.Size([31092]).
    size mismatch for cls.predictions.decoder.weight: copying a param with shape torch.Size([31090, 768]) from checkpoint, the shape in current model is torch.Size([31092, 768]).
    size mismatch for cls.predictions.decoder.bias: copying a param with shape torch.Size([31090]) from checkpoint, the shape in current model is torch.Size([31092]).
    You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.

only when i add 'ignore_mismatched_sizes=True', it works.

2. the self.args.task in knowledge.py

if self.args.task == 'pretrain' or self.args.task == 'retrieval':

but there is no task in the args

3. GPU number the --n_gpu=1 ,and error here is

RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

I don't know if it is a mistake in the code. I hope you can help me confirm it. Thanks in advance~

[mlii0117]

Hi, sorry for the inconvenience. We have noticed these issues and will update the code soon.

[5k5000]

the same questions, looking forward to updates.

[NorthTree22]

I meet the same question, and I want to know ,if the code support undistributed training?

[mlii0117]

@NorthTree22 Hi, it does support. I will update the code soon.

[Eldo-rado]

@NorthTree22 👋Hey! I have met you in other repositories related to report generation. Could you please leave your email address so that we can exchange some relevant content. 😆

[NorthTree22]

@Eldo-rado Ok, that would be great. You can contact me at 2874338756@qq.com

[NorthTree22]

@NorthTree22 Hi, it does support. I will update the code soon.

Thank you very much. Look forward to your update.

[lq-blackcat]

I meet the same question

[mk-runner]

I meet the same question

[Junyiliu0]

how to solve the problem 'unmatched weight'? Only 'ignore_mismatched_sizes=True'?

[Junyiliu0]

how to solve it? if self.args.task == 'pretrain' or self.args.task == 'retrieval':?

[Eldo-rado]

how to solve it? if self.args.task == 'pretrain' or self.args.task == 'retrieval':?

Please do not try my solutions 😂, as they are just simple attempts and are not succeed in running the code. Please wait for an update from the author.

[Junyiliu0]

My email is 337940719@qq.com. I have solve the problems you can contact me~ 😊

[chauncey-tow]

My email is 337940719@qq.com. I have solve the problems you can contact me~ 😊

@Junyiliu0 Hi, did you solve the following problem? Can you share your solution? if self.args.task == 'pretrain' or self.args.task == 'retrieval':?

[shiroi2333]

I also want to know how to solve it? if self.args.task == 'pretrain' or self.args.task == 'retrieval':?

[shiroi2333]

I meet the same question