search

mutonix / pyramidinfer

31 stars 0 forks source link

window_len == 1 in generation phase #1

Closed cyLi-Tiger closed 4 months ago

cyLi-Tiger commented 5 months ago

Thanks for the great work!

here, the window_len == 1, is it expected?

Besides, why evict tokens from -(1 + gen_recent_length + exceed_length_to_compress):-(1 + gen_recent_length) but not -(gen_recent_length + exceed_length_to_compress):-(gen_recent_length)? What's the point of 1 here?

mutonix commented 4 months ago

Many thanks for your meticulous inspection. It is a bug we missed during the code reorganization and we fixed it in the latest commit in here.

The 1 here added to recent_length is a legacy behavior from earlier code, which has no influence and can be deleted. Keeping it will only result in a longer window length increased by 1.