To run the project

chmod +x run.sh
./run.sh

Dependencies

TagMe: pip install tagme (GCUBE_TOKEN = "bfbfb535-3683-47c0-bd11-df06d5d96726-843339462")
Argparse: conda install -c anaconda argparse=1.3.0
Copy: For making deepcopy
Itertools: conda install -c auto more-itertools=2.2
TrecCar: https://github.com/TREMA-UNH/trec-car-tools/tree/master/python3

Following results will appear once the shell script is run.

============================================
Generating results file with regular queries
============================================
Gathered Pages
Gathered Paragraphs
Gathered Regular Queries
Gathered Results

=======================================
Running evaluation framework...........
=======================================
runid    all    test
num_q    all    60
num_ret  all    2100
num_rel_ret  all    174
map  all    0.2295310245310245
gm_map   all    0.6585904496408844
Rpec     all    0.10599206349206347
recip_rank   all    0.22433261183261183
P_5  all    0.07
P_10     all    0.04500000000000001
P_15     all    0.03333333333333333
P_20     all    0.026666666666666675
P_30     all    0.018333333333333333
P_100    all    0.005500000000000001
P_200    all    0.0027500000000000007
P_500    all    0.0011000000000000003
P_1000   all    0.0005500000000000001

=================================================================
Generating results file with tagme enchanced queries.........
=================================================================
Gathered Pages
Gathered Paragraphs
Generating TagMe enchanced queries........
Gathered Tagme Enhanced Queries
Gathered Results
=================================================================
Running evaluation framework on results on tagme enhanced results
=================================================================
runid    all    test
num_q    all    60
num_ret  all    2100
num_rel_ret  all    174
map  all    0.2079354904354904
gm_map   all    0.6511606087084545
Rpec     all    0.11154761904761905
recip_rank   all    0.2138481888481888
P_5  all    0.07666666666666667
P_10     all    0.04666666666666667
P_15     all    0.03333333333333333
P_20     all    0.026666666666666675
P_30     all    0.018333333333333333
P_100    all    0.005500000000000001
P_200    all    0.0027500000000000007
P_500    all    0.0011000000000000005
P_1000   all    0.0005500000000000002