Open oublalkhalid opened 3 months ago
Hi @oublalkhalid, thanks for the issue. We are planning to release the code for finetuning and reproducing the results in the paper by the next week (by end of march).
Hi @oublalkhalid Can you please share additional information about the results? You can reach me by email too if you'd like (arjun [dot] ashok @ servicenow [dot] com)
Hi @oublalkhalid Please check the repo now. All scripts have been released.
Hi @ashok-arjun
When running the pretrain.sh scripts I am always getting this error, and I tried with different datasets
File "/lag-llama/data/data_utils.py", line 40, in __next__
(dataset,) = self._rng.choices(self._datasets, weights=self._weights, k=1)
File "/.conda/lib/python3.10/random.py", line 533, in choices
total = cum_weights[-1] + 0.0 # convert to float
IndexError: list index out of range
Hello, thank you for sharing the initial code. I aim to replicate the experiment outlined in your paper before proceeding to fine-tune the model with additional datasets. Could you kindly assist me in accessing a tutorial on reproducing the experiment results? Currently, the model seems to lack convergence, so any guidance you could provide would be greatly appreciated.
Once again, thank you for your efforts.
best, koublal