search

opentargets / issues

Issue tracker for Open Targets Platform and Open Targets Genetics Portal
https://platform.opentargets.org https://genetics.opentargets.org
Apache License 2.0
12 stars 2 forks source link

Review of our OpenAI based API #3249

Closed mbdebian closed 2 months ago

mbdebian commented 3 months ago

The backend MIG running our OpenAI API service, detailed here has, apparently, a constant traffic of 0 requests.

We need to verify with the platform analytics whether this service is actually being used and, in case it is, check the usage profile to verify that this backend receives no traffic from the global load balancer due to the configured CDN (if active).

The @opentargets/data-team and @opentargets/fe-team can also provide insights from the OpenAI API token usage by looking into our OpenAI dashboard details.

ireneisdoomed commented 3 months ago

The dashboard shows a mild usage of the app: ~600 requests in Feb, 300 in January, none in January. Moreover, the use is not constant, but rather in peaks.

image

Since we developed this, Langchain has launched a deployment service called Langserve that integrates with lambda functions in GCP. I haven't worked with it, but it is an alternative BE you might want to take into consideration.