search

SmartAPI / smartAPI

Intelligent APIs for a more connected web: https://smart-api.info
MIT License
21 stars 16 forks source link

Harvest metakg from the supported APIs and provide an endpoint to query aggregated metakg #179

Open newgene opened 1 year ago

newgene commented 1 year ago

Translator APIs in SmartAPI registry provide meta-knowledgegraph (or metakg) information. These APIs can be TRAPI APIs or those APIs annotated by x-bte extensions. We can regularly harvest metakg info from these APIs and index it in Elasticsearch, and then provide a searchable endpoint to query metakg info.

The existing /api/metakg endpoint is a proxy to http://api.bte.ncats.io/metakg, which holds all metakg in memory. As metakg grows bigger, the current implementation won't scale up, and its search capacity is limited too.

We plan to move this metakg functionability from BTE to SmartAPI, and provide a native metakg API endpoint.

newgene commented 1 year ago

This is related to this BTE issue: https://github.com/biothings/BioThings_Explorer_TRAPI/issues/537

Once this feature is implemented in SmartAPI, we will evaluate this BTE issue to bring the same feature into BTE (e.g. by calling SmartAPI's native /api/metakg endpoint)

newgene commented 1 year ago

metakg harvesting can be adopted from:

https://github.com/newgene/bte-py/tree/main/biothings_explorer/smartapi_kg

but should be much simplified for the parts needed here.