search

osirrc / jig

Jig for the Open-Source IR Replicability Challenge (OSIRRC)
13 stars 3 forks source link

Unify default evaluation settings, evaluate on all topics. #106

Closed arjenpdevries closed 5 years ago

arjenpdevries commented 5 years ago

Since the OSIRRC "default evaluation" reports P@5 instead of P@30, I suggest to modify the jig default setting to reflect this.

I also added the -c flag to ensure that evaluation measures are reported on all topics and not just those that the systems returned results for.

Re: https://github.com/osirrc/jig/issues/105

arjenpdevries commented 5 years ago

Hold on, this misses one commit.