search

csag-uct / Metadata-Harmonisation-Tool

Health Data Harmonisation Platform
Creative Commons Attribution Share Alike 4.0 International
3 stars 6 forks source link

Remove need for API Keys #15

Open peterm790 opened 5 months ago

peterm790 commented 5 months ago

We really on OpenAI API calls in two key areas:

1) To generate the distributions we create a prompt and then send it to the API for a response. Ideally we could just swap out this API call for a call to a new function which runs a Llama model locally. I think https://github.com/abetlen/llama-cpp-python is ideal for this https://github.com/csag-uct/Metadata-Harmonisation-Tool/blob/a6e75c5b2bc31d50027093dcc7547c9bfb91442e/app/components/generate_descriptions.py#L116

2) To generate the embeddings we rely upon the OpenAI embeddings model in a few places. Presumably llama-cpp must be capable of providing some kind of embedding or ideally someone has built an open source alternative on top of it. I looked a while back but cam up short. https://github.com/csag-uct/Metadata-Harmonisation-Tool/blob/a6e75c5b2bc31d50027093dcc7547c9bfb91442e/app/components/get_recommendations.py#L19

peterm790 commented 2 months ago

https://github.com/peterm790/Health_Data_Harmonisation_Platform/tree/add_local_model_support - an attempt at this, not sure it is worth pursuing as the API differences between models make universal support tricky

peterm790 commented 2 months ago

https://github.com/csag-uct/Metadata-Harmonisation-Tool/pull/18