search

IBM / kgi-slot-filling

This is the code for our KILT leaderboard submissions (KGI + Re2G models).
Apache License 2.0
147 stars 14 forks source link

run jar wrong #1

Open wanghuanOdile opened 2 years ago

wanghuanOdile commented 2 years ago 
java com.ibm.research.ai.pretraining.retrieval.DPRTrainingData \
> -passageIndex anserini_passage_index \
> -positivePidData trex_train_positive_pids.jsonl \
> -trainingData trex_dpr_training_data.jsonl

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/lucene/search/Query at com.ibm.research.ai.pretraining.retrieval.DPRTrainingData.main(DPRTrainingData.java:99) Caused by: java.lang.ClassNotFoundException: org.apache.lucene.search.Query at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ... 1 more

hi, I have prepared the data index and trex data, when I run the jar to search data, I got wrong. Are there any problems?

But I run search in pyserini(python version of anserini), I can get results. Can you share what the jar does so I can reproduce in python? Thanks a lot!

from pyserini.search import SimpleSearcher

searcher = SimpleSearcher('anserini_passages_index')
hits = searcher.search('document')

for i in range(len(hits)):
    print(f'{i+1:2} {hits[i].docid:4} {hits[i].score:.5f}')
michaelrglass commented 2 years ago

This seems like a classpath issue. Can you confirm that your CLASSPATH has the anserini-0.4.1-SNAPSHOT-fatjar.jar and dprBM25.jar? Like: export CLASSPATH=jar/dprBM25.jar:Anserini/target/anserini-0.4.1-SNAPSHOT-fatjar.jar