Weixin-Liang / LLM-scientific-feedback

Can large language models provide useful feedback on research papers? A large-scale empirical analysis.
Creative Commons Attribution 4.0 International
489 stars 49 forks source link

parsing error #24

Open harrywang opened 9 months ago

harrywang commented 9 months ago

parsing error with the following paper: 2210.14896.pdf

Failed to generate review... Error: Encountered text corresponding to disallowed special token '<|endoftext|>'.
If you want this text to be encoded as a special token, pass it to `allowed_special`, e.g. `allowed_special={'<|endoftext|>', ...}`.
If you want this text to be encoded as normal text, disable the check for this token by passing `disallowed_special=(enc.special_tokens_set - {'<|endoftext|>'})`.
To disable this check for all special tokens, pass `disallowed_special=()`.