ASUS-AICS / LibMultiLabel

A library for multi-class and multi-label classification
MIT License
152 stars 30 forks source link

Batch NDCG #342

Closed donglihe-hub closed 10 months ago

donglihe-hub commented 10 months ago

What does this PR do?

The original NDCG metric calculates scores for one instance at a time, which is inefficient. The new NDCG metric calculate scores in batch.

Performance Test Settings:

Number of labels = 100

Batch size = 40 Number of batchs = 100 Effective number of validation samples = 4000

Results:

1c07d1925b9145c64f09d44690f1112

Test CLI & API (bash tests/autotest.sh)

Test APIs used by main.py.

Check API Document

If any new APIs are added, please check if the description of the APIs is added to API document.

Test quickstart & API (bash tests/docs/test_changed_document.sh)

If any APIs in quickstarts or tutorials are modified, please run this test to check if the current examples can run correctly after the modified APIs are released.

donglihe-hub commented 10 months ago

We should review the use of plurals vs singulars.

A doubt that has haunted me for years is that why it is preds and target, rather than preds and targets or pred and target. Therefore, I prefer using singular form for all variables except for those concepts that have been accepted by the general public.