请问如何debug下每次中间是哪些token位置被选中进行拼接的？

thunlp / InfLLM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

MIT License

306 stars 29 forks source link

Closed yinochaos closed 7 months ago

yinochaos commented 7 months ago

RT 请问如何debug下每次中间是哪些token位置被选中进行拼接的？能否告诉下相关代码的位置

guyan364 commented 7 months ago

你可以查看 get block topk 的输出 https://github.com/thunlp/InfLLM/blob/main/inf_llm/attention/context_manager.py#L366，位置即为 [n_init + id block_size, n_init + id block_size + block_size]