search

NCATSTranslator / Ordering-Organizing

ARS Level O&O
0 stars 0 forks source link

ARS and Subsystem performance frequently results in no scores #58

Open sharatisrani opened 3 weeks ago

sharatisrani commented 3 weeks ago

As requested by TACT members: this is a tracking issue that subsumes many, including new vs old confidence scoring, and new novelty factor scores, analysis of grouping. ARS calls Appraiser which calls other scoring systems as in slide 2 of O&O flow diagram.

It is actually a showstopper issue until the scores flow with reasonable certainty, guessing 90%.

At the moment, there are two suspects for slowness - networking issues that affect the Appraiser-ARS link, and PubMed Reference performance in returning recency results to Novelty scoring.

maximusunc commented 2 weeks ago

Answer Appraiser has been updated to disable novelty scoring, as well as other performance enhancements. It is now returning the other scores within seconds instead of minutes. I say we run a few tests, but otherwise this is no longer an issue.

bill-baumgartner commented 2 weeks ago

Just a note regarding the Publication Metadata API -- it was designed to meet specs provided by the UI team and therefore may not be suitable for the recency filter calculations if the issue is speed. A dramatic speedup would likely require a different underlying architecture (probably in-memory) and trimming down the returned data to only include only PMID and date published.

In case it is useful for testing/developing the recency calculation, I have generated a mapping from PMID to year published in this file.