Fine Tuning Splitting.

[BITEEE0308]

Hi Mike,

I'm wondering in terms of fine-tuning, are we going to do all openai models or just Davinci? Also, in terms of the 50:50 splitting on the quiz data, how do you want us to do it if not just randomly select 50 percent as training data? (I think you mentioned it in the meeting but I didn't quite follow it at the time. )

[mikeizbicki]

I'm wondering in terms of fine-tuning, are we going to do all openai models or just Davinci?

I'm not sure. We'll start with the davinci model and go from there.

Also, in terms of the 50:50 splitting on the quiz data, how do you want us to do it if not just randomly select 50 percent as training data? (I think you mentioned it in the meeting but I didn't quite follow it at the time. )

We don't want a 50:50 split. We want exactly 5 questions in the training data and the rest in the test data. The test data will be a different size for each quiz, but the training data will be exactly the same size. Then we can try 0, 1, 2, 3, 4, and 5 shot learning.

For consistency's sake, just use the first 5 questions as the training data.