Closed yonromai closed 11 months ago
(Note: Please discard the first commit, it belongs to the already open PR https://github.com/related-sciences/nxontology-ml/pull/25 - I'll rebase once merged)
@yonromai for future reference you can change the base of a PR:
You would change the base to the branch of the other PR, and carry on :D. GH will even rebase this PR once you merge the other one.
(^ Rebased on main after merging #25)
The updated prompt with the additional examples prompts/precision_v2.txt
LGTM
Gist: Adding the new examples surprisingly didn't improve the performance of the model.
Good to know, though a big confounder with this is that comparing the GPT4 results to the manual labels directly is somewhat ill-defined. I would still expect improvements/regressions to be detectable this way if they're above a certain magnitude, so the best conclusion for my money is that the examples didn't change anything or helped to a undetectably small extent.
Good to know, though a big confounder with this is that comparing the GPT4 results to the manual labels directly is somewhat ill-defined. I would still expect improvements/regressions to be detectable this way if they're above a certain magnitude, so the best conclusion for my money is that the examples didn't change anything or helped to a undetectably small extent.
Sounds good to me. My gut feeling tells me that it's good to keep that extra bit of supervision so I'd be in favor of merging the prompt with the added extra examples. I'll just update the older prompt to include the extra example and use that updated prompt to add the GPT tags to the catboost model (in a different PR).
update the older prompt to include the extra example and use that updated prompt to add the GPT tags to the catboost model (in a different PR).
Perfect 👍. That sounds right to me.
@eric-czech
Please take a look at:
prompts/precision_v2.txt
Gist: Adding the new examples surprisingly didn't improve the performance of the model.
Notes:
cc: @dhimmel @ravwojdyla