search

Refefer / fastxml

FastXML / PFastXML / PFastreXML - Implementation of Extreme Multi-label Classification
Other
149 stars 47 forks source link

Run benchmarks against the library #5

Closed AlexGonRo closed 6 years ago

AlexGonRo commented 6 years ago

There is a TODO bullet point in the README file that reads: "Run all the standard benchmark datasets against it."

I was planning on using this framework for one of my courses at university. As a part of my small project, I will be running the FastXML and pFastreXML algorithms against (all?) the benchmark datasets that the papers use (I basically need a baseline for my results).

Because of this, I would like to know if you still want to test the application and, if so, which data do you need. I might be able to give you a hand with it.

Refefer commented 6 years ago

That'd be wonderful!

Really, we need few things: Hyperparameters and Prec/NDCG/PsNDCG. The former is especially important since we're using a different optimizer implementation than the original paper (who simply embedded liblinear directly in the code), so that'll be useful. Ideally, having the scripts that run the benchmarks would be great, but not strictly necessary.

We'll also need to fix the PsNDCG that's baked into FastXML's metrics - it's incorrect currently since it isn't bounded from [0,1]. That should be a fairly quick task also.

Refefer commented 6 years ago

I'm going to close this for the time being. Let me know if you're still planning on working on this.