search

kubetoolsca / krs

A GenAI-powered Kubetools Recommender System
https://kubetools.io
Apache License 2.0
91 stars 20 forks source link

[Feature]: support LocalAI backend #54

Open karanveersingh5623 opened 3 months ago

karanveersingh5623 commented 3 months ago

Is this feature request related to a problem? None

Problem Description No response

Solution Description support localai.

Benefits The LocalAI can make it easier for users to interact with KRS.

karanveersingh5623 commented 3 months ago

@OluchukwuON

abhimazu commented 3 months ago

@OluchukwuON --> Find target open sourced models that can perform the task of log analysis and respond back. Search huggingface for target models.

Criteria: 1) They should be under 4 GB in size 2) They should be able to understand Kubernetes context to respond with resolutions 3) They should be chat models that can respond

Tasks: 1) List all candidate models in a csv file with their size, and urls from huggingface 2) Create a list of 10 sample questions with faulty Kubernetes Pod logs that you will use to benchmark these models 3) Test all these models using the questions by creating end points on cloud instances provided by @karanveersingh5623 one by one using either langchain, TGI or tensorflow serving. 4) Record the individual model responses for all the 10 questions, and record the execution time for each question inside a spreadsheet, the target is to find the WPS speed of each model (word per second) when deployed locally.