Open mohsenhariri opened 1 month ago
Let me check that.
Currently this version of code does not support Mistral yet. However you can try it with Llama3 and Llama2. Support of Mistral would be added soon.
Currently this version of code does not support Mistral yet. However you can try it with Llama3 and Llama2. Support of Mistral would be added soon.
Does this mean the current version is not ready to reproduce the GEAR on Mistral models, as reported in the paper draft? If this is not the case, could you provide some suggestions since I failed to find related shell scripts.
Thanks!
@HaoKang-Timmy
Title: Evaluation Code Produces Identical Results with Different Caching Methods
Description:
It seems the evaluation code leads to the same result with different caching methods. I used these models:
mistralai/Mistral-7B-v0.1
mistralai/Mistral-7B-Instruct-v0.2
with 3 different caching methods:
--compress_method KCVT
,--compress_method GEAR
, and--compress_method KIVI_V2
. In all cases, the result is:Steps to Reproduce:
mistralai/Mistral-7B-v0.1
andmistralai/Mistral-7B-Instruct-v0.2
.--compress_method KCVT
,--compress_method GEAR
, and--compress_method KIVI_V2
.Expected Behavior: Different caching methods should produce varying results in KIVI accuracy.
Additional Information: I checked the input arguments and the evaluation script reads them correctly, so I am sure I had different setups.