All metrics are NaN

[laura-dietz]

[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.284s [INFO] Finished at: Wed Mar 15 18:11:47 EDT 2017 [INFO] Final Memory: 40M/1523M [INFO] ------------------------------------------------------------------------ Indexing Paragraphs Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 2 at edu.unh.cs.trec.TrecCar.main(TrecCar.java:52) Metric Precision@5 = NaN err: NaN Metric Precision@10 = NaN err: NaN Metric Precision@15 = NaN err: NaN Metric Precision@20 = NaN err: NaN Metric Precision@100 = NaN err: NaN Metric R-Precision = NaN err: NaN Metric MAP = NaN err: NaN Metric Mean Reciprocal Rank = NaN err: NaN dietz@jelly:~/datasci/code2/team3$

[gummibearehausen]

you can try https://github.com/tuckerowens/trec-car-quick-start

(1) chmod +x easy_run.sh (2) ./easy_run.sh $1 $2 $3 $1 is [cbor.paragraph] $2 is[cbor.outline] $3 is [cbor.qrel] you can try using files in the test200 v1.4.

[laura-dietz]

I got it running , but I think there is something wrong with your metrics.

Can you tell me where the trec-eval compatible file gets written to (so I can double check)?

[tuckerowens]

The runfile is written to a temp file. The result of the mktemp command. There are two options:

• Go to /tmp and use ll -rt to find the most recent files. One of those is likely the run file.
• Edit the easy_run script to use a permanent file instead.