Closed tuongtranngoc closed 1 month ago
You can use the file test_with_oracle_prefetched_ids_for_reranking.json in the shared dataset on google drive. It is a list of Json string, each line contain a Json string of a data example. The "prefetched_ids" contains 2000 BM25 prefetched candidates and the actually cited id "positive_id" is also manually added into the prefetched_ids list. With this file you do not need to run the prefetch, but just run the reranker as shown at https://github.com/nianlonggu/Local-Citation-Recommendation?tab=readme-ov-file#get-paper-recommendations
Thanks @nianlonggu , I used the json that you mentioned, but I got error when loading it: json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 24088) Can you help me check it?
You can read the file line by line, and convert each line to a dictionary by json.loads()
Thanks @nianlonggu, I got it
So, how to produce reranking performance as below results:
I do not find guideline in README
You can refer to the last section in https://github.com/nianlonggu/Local-Citation-Recommendation/blob/main/Turorial_Local_Citation_Recommendation.ipynb
Sorry @nianlonggu , as I know, the last section is evaluated entire Recommendation Pipeline both prefetched and reranking. I only wanna to evaluate reranker that you mentioned in paper with table 3
Yes. But you can easily customize it by replacing the prefetched IDs with the "prefetched_ids" I mentioned in the previous Json file.
thanks @nianlonggu , I got it
@nianlonggu How to evaluate only reranking module without running pre-fetched candidate?