Add some more few shot examples

[yonromai]

@eric-czech

I do think we can add these misclassified examples to the prompt though:

Please take a look at:

• The updated prompt with the additional examples prompts/precision_v2.txt
• The related results in "section 5" of the updated notebook.

Gist: Adding the new examples surprisingly didn't improve the performance of the model.

Notes:

• I added some prompts and completions as gists for manual inspection
• I was worried about previous global scope polluting the section 5 results so I restarted the kernel and ran only section 5; there results were identical

cc: @dhimmel @ravwojdyla

[ravwojdyla]

(Note: Please discard the first commit, it belongs to the already open PR https://github.com/related-sciences/nxontology-ml/pull/25 - I'll rebase once merged)

@yonromai for future reference you can change the base of a PR:

You would change the base to the branch of the other PR, and carry on :D. GH will even rebase this PR once you merge the other one.

[yonromai]

(^ Rebased on main after merging #25)

[eric-czech]

The updated prompt with the additional examples prompts/precision_v2.txt

LGTM

Gist: Adding the new examples surprisingly didn't improve the performance of the model.

Good to know, though a big confounder with this is that comparing the GPT4 results to the manual labels directly is somewhat ill-defined. I would still expect improvements/regressions to be detectable this way if they're above a certain magnitude, so the best conclusion for my money is that the examples didn't change anything or helped to a undetectably small extent.

[yonromai]

Good to know, though a big confounder with this is that comparing the GPT4 results to the manual labels directly is somewhat ill-defined. I would still expect improvements/regressions to be detectable this way if they're above a certain magnitude, so the best conclusion for my money is that the examples didn't change anything or helped to a undetectably small extent.

Sounds good to me. My gut feeling tells me that it's good to keep that extra bit of supervision so I'd be in favor of merging the prompt with the added extra examples. I'll just update the older prompt to include the extra example and use that updated prompt to add the GPT tags to the catboost model (in a different PR).

[eric-czech]

update the older prompt to include the extra example and use that updated prompt to add the GPT tags to the catboost model (in a different PR).

Perfect 👍. That sounds right to me.