search

Zoeyyao27 / SirLLM

This repository contains the code for the paper: SirLLM: Streaming Infinite Retentive LLM
55 stars 3 forks source link

Question about "KV cache is reset to zero" #1

Closed YukiChen-yuxin closed 4 months ago

YukiChen-yuxin commented 4 months ago

Hi, the SirLLM is awesome! Just curious about one point, you mentioned "DailyDialog and Grocery Shopping datasets, where the KV cache is reset to zero after each round", then how does LLM remember the info about the grocery list in the final round if the KV cache is reset. Thanks!

Zoeyyao27 commented 4 months ago

Hello! Thank you for your interest in our work. In our paper, a round consists of multiple turns of dialogue. Figure 8 illustrates a round of dialogue in the Grocery Shopping dataset. Please feel free to check it out.

YukiChen-yuxin commented 4 months ago

Got it, thanks!