search

StonyBrookNLP / ircot

Repository for Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions, ACL23
https://arxiv.org/abs/2212.10509
Apache License 2.0
173 stars 21 forks source link

Data and trained model #6

Closed minjunp closed 1 year ago

minjunp commented 1 year ago

Hi,

I have several questions regarding your work!

  1. It seems 2wikimultihopqa is not properly downloaded in raw_data.sh
  2. In the code, are you saving the trained model with best hyperaparmeters?
  3. What's the use of base_configs and instantiated_configs folders?

Thank you in advance.

HarshTrivedi commented 1 year ago
  1. Maybe your process stopped at some point as running ./download/raw_data.sh downloads lots of things. Have you tried running the relevant section of the script for 2wikimultihopqa individually? I just tried it, and it's working fine for me. If you download files individually, just make sure that the directory structure looks like what's given in the comments of the script (at the end) and you're good.
  2. There is no real training anywhere in this work. It's all based on few-shot/incontext demonstrations. So we don't need to save the trained model weights. However, we do search for hyperparameters and select them based on a small dev set. The configs corresponding to these HPS and the one selected as the best one will be stored in one of the intermediate steps.
  3. For each config in base_configs, we do HP/grid search (configured here). The configs corresponding to this grid search (i.e., configs instantiated with the given set of HPs) are stored in instantiated_configs.
minjunp commented 1 year ago

Thank you!