Samagra-Development / ai-tools

AI Tooling to bootstrap applications fast
41 stars 109 forks source link

Fine tuning the classifier for new schemes #319

Closed GautamR-Samagra closed 1 month ago

GautamR-Samagra commented 1 month ago

SAFAL-SUGAM Schemes Mapping.xlsx Above is a dataset for schemes in Odisha.

Below are some Schemes for Rajasthan : 'Diggi' 'Jalnidhi' 'PM-Kisan' 'PM-Kissan' 'Jalhauz'

For each of the schemes, create questions like :

Do not use the word scheme in these questions (classifier works well if the word scheme is mentioned. retrain the bert-base-uncased and the distilbert classifier (in that order) for this additional dataset and log the eval scores.


some other additions : # Query Ideal Classifier Classified as
1 Tell me about PM kisan Scheme Weather
2 Dealers in undoo POP- dealer Non-Agri
3 Where can i get machinery in barmer POP- dealer Non-Agri
4 Tell me about Diggi scheme Scheme Pest
5 Tell me about solar pump Agri Seed
6 Tell me about solar pump set Scheme Seed
7 Tell me about PM Kusum Scheme Non-Agri
8 Tell me about shade net scheme Scheme Non-Agri
GautamR-Samagra commented 1 month ago

@Jiya126