Closed Michelleable closed 4 months ago
Hi, there are some differences between the longbench indicator I ran and the paper. Is there any wrong setting in my experiment? Thanks.
model: type: inf-llm path: Mistral-7B-Instruct-v0.2 fattn: false block_size: 128 base: 1000000 distance_scale: 1.0 n_init: 128 n_local: 4096 topk: 16 repr_topk: 4 max_cached_block: 32 exc_block_size: 512 async_global_stream: true cache_strategy: lru faiss: false
max_len: 2147483647 truncation: suffix chunk_size: 8192
conv_type: mistral-inst
Hi, there are some differences between the longbench indicator I ran and the paper. Is there any wrong setting in my experiment? Thanks.
model: type: inf-llm path: Mistral-7B-Instruct-v0.2 fattn: false block_size: 128 base: 1000000 distance_scale: 1.0 n_init: 128
n_local: 4096 topk: 16
repr_topk: 4 max_cached_block: 32 exc_block_size: 512 async_global_stream: true cache_strategy: lru faiss: false
max_len: 2147483647 truncation: suffix chunk_size: 8192
conv_type: mistral-inst