search

acikyazilimagi / depremadres-intent-classification-v0

37 stars 26 forks source link

Yeni bert classifier'i ekleyip service ve main binarylerine ekler. #31

Closed telecastinator closed 1 year ago

telecastinator commented 1 year ago

Yeni bert classifieri classifier interfaceinde class olarak yaptim, api servis ve main.py ye bagladim, commandline flaglere bagladim (hangi classifer calissin diye) ve evale ekledim.

Eski eksik eval datasiyla kurtarma bert classifierda performans soyle:

Intent: KURTARMA

KURTARMA Precision: 0.62 KURTARMA Recall: 1.00 KURTARMA F1: 0.77

Label eksikliginden geliyordur FPler ama ayni kosullarda kalsilastiirlinca rule based classifier hala karsilastirilabilir derecede iyi.

Closes #30

telecastinator commented 1 year ago

If there is any strict policy about not uploading data files, eval.csv should be removed. We also started using a new evaluation set covering examples from all classes. Merve posted it to AI channel last night.

You're right, but I'm not aware of such policy, and the presence of other files in this data folder indicates it seems OK.

I know this eval file is dated and missing intents, and it is only provisionally here to have a workihng eval.py, I think it can stay.

We should definitely use the newest eval sets (as they get updated). Not sure if we'll keep keeping there here or hook up to a huggingface API.

telecastinator commented 1 year ago

Oops mistakenly closed the PR.

zafercavdar commented 1 year ago

A general comment: is it possible to reduce the number of main.py files?

telecastinator commented 1 year ago

A general comment: is it possible to reduce the number of main.py files?

I'm also not sure they exist but here's how I made sense; app_main.py is the main for the service, main.py is the command line version for quickly checking that the classifiers work. E.g 'python3 main.py --text="yardim"'. A quick way of integration testing of the models without needing to issue properly formed rest queries.

telecastinator commented 1 year ago

Need to fast convesation with this two PR. @ediziks @zafercavdar

29

This looks great, I think there are no conflicts as is. Once #29 and #31 is merged, we can do a new PR to add a "KoalaClassifier" wrapper to the ml_modules and add it as a classifier option in the main service binary.

telecastinator commented 1 year ago

Oops, looks like max 2 reviewers are allowed.

yunusemreemik commented 1 year ago

@ediziks @zafercavdar @telecastinator When you're available. I'll be waiting on discord/indent-classification.