We are using your repo to reproduce the IR-CoT. I noticed there seems to be caching that memorizes LLM results.
When I rerun experiments with the same configuration settings (such as ircot_qa with flan-t5-xl for 2wikimultihopqa), the LLM server finishes very quickly, indicating it might be using cached results.
I tried making 'skip_if_exist' arg false, but it still seems to use the cache or logs. How can I turn off this caching feature?
Thank you for your awesome repository.
We are using your repo to reproduce the IR-CoT. I noticed there seems to be caching that memorizes LLM results.
When I rerun experiments with the same configuration settings (such as ircot_qa with flan-t5-xl for 2wikimultihopqa), the LLM server finishes very quickly, indicating it might be using cached results.
I tried making 'skip_if_exist' arg false, but it still seems to use the cache or logs. How can I turn off this caching feature?