search

inspirehep / magpie

Deep neural network framework for multi-label text classification
MIT License
684 stars 192 forks source link

Parameterize the top_k_categorical_accuracy metric #172

Open somnathrakshit opened 5 years ago

somnathrakshit commented 5 years ago

Modify to have parameterized version of accuracy metric

jstypka commented 5 years ago

Hey @somnathrakshit, thanks for the PR!

This change makes parametrisation of the top_k_accuracy metric possible, but doesn't enable us to do anything else. The way I thought about it, is to have the top_custom_categorical_accuracy function in the user code, outside of Magpie and enable a field called metrics=[] in the Magpie API to pass arbitrary functions to be computed. That way users can compute whatever they want. We should also enable some sensible defaults to be computed when nothing is passed in.

Hope that makes sense!

somnathrakshit commented 5 years ago

Yes, it does. I will try to do it over the weekend but not sure if I will get time or not.