Closed RManLuo closed 4 months ago
The same problem happens to me.
👋 Hello,
Thank you for bringing this issue to our attention. Regarding the problem you've been encountering with the model generating gibberish, I believe I've identified the true solution.
The root cause of the issue seems to be related to the setting add_special_tokens=False in the above code. To resolve this problem, we should set add_special_tokens=True. After set add_special_tokens=True, the model output meets the expectation, and the setting of this parameter in generate is consistent with that in the process of our train. In our experiment, add_special_tokens is also set to True, and the open source code does not modify this parameter. We have corrected this part of the code and resubmitted it.
Setting add_special_tokens=True is crucial when using our model for text generation. This parameter determines if special tokens, such as the starting [CLS] token and ending [SEP] token for some models, are added to the beginning and end of the sequence. These special tokens are vital for the model to correctly understand the context of the input.The add_special_tokens parameter ensures that these important tokens are correctly added to the input sequence, enabling the model to interpret and process the data accurately. The absence of these special tokens may result in the model failing to properly understand the structure of the input data, which affects its performance and the quality of the generated output, possibly leading to the gibberish output you've encountered.
Thanks a lot for the reply. After updating the code to this commit 1cfa113294427fe1081ac0d5906e701eee31305c, the issue is fixed!
Hi, thanks for open-sourcing this insightful project. However, I encounter some issues when reproducing the results.
I loaded the checkpoint and ran the
generate.py
. I found that theresponse0
which is generated from the node embedding is a bunch of meanless texthttps://github.com/alibaba/GraphTranslator/blob/bf7c4469dd66fa9e3ece044cf99e0abbacc11620/Translator/models/translator_models/translator_chatglm_arxiv.py#L258-L264
Here is an example result I got Input:
Response0:
Then I found that you insert the content of response0 into the final prediction prompt (
summary_prompt
) for prediction, which looks likesummary_prompt:
The output, on the other hand, looks reasonable
Response2:
I wonder if I made any mistakes during the inference. Why is 'response0' filled with meaningless text? How can the language model still produce a coherent output with such nonsensical context?
I am really looking forward to hearing from you. Thanks again for publicizing your project.