search

LoveCatc / supervised-llm-uncertainty-estimation

This repo contains code for paper: "Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach".
10 stars 0 forks source link

Model weights #3

Closed simoneVU closed 3 months ago

simoneVU commented 3 months ago

Really interesting paper! I was interested in whether you have in plan to release the model weights for random forest any soon.

Thank you!

ly-liu15 commented 3 months ago

Thank you for your kind words!

For each LLM model, each feature selection way (middle layer/last layer, question/answer, average/last token), and each dataset, we should train a random forest model, so the total number of models is large. That's the reason we don't plan to release the model weights here.

But in line 214-237 of supervised_calibration.py, we provide the hyperparameters of the random forest model including the random state, so the weights are available if you can use them to train it.

simoneVU commented 3 months ago

Thank you very much for the quick response!