search

THUDM / LongCite

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA
Apache License 2.0
243 stars 16 forks source link

about the maxium length. #8

Open quzx opened 6 days ago

quzx commented 6 days ago

The model has 128k sequence_length, 1M token how to transfer to it? We need to split the source text? Thanks.

Neo-Zhangjiajie commented 6 days ago

Yes, you may need to split the source text or use retrieval first to fit the maximum sequence length.