intelligentnode / IntelliNode

Access the latest AI models like ChatGPT, LLaMA, Diffusion, Gemini Hugging face, and beyond through a unified prompt layer and performance evaluation
https://show.intellinode.ai
Apache License 2.0
215 stars 14 forks source link

Multi model evaluation feature #39

Closed Barqawiz closed 11 months ago

Barqawiz commented 11 months ago

add auto evaluation function by receiving list of providers and text to evaluate.

await llmEvaluation.compareModels(inputString, targetAnswers, providerSets);

output example:

  openai: [
    {
      prediction: "Sure! Photosynthesis is the process by which plants use sunlight, water, and carbon dioxide to make their own food. First, the plant's leaves capture sunlight. Then, through a special pigment called chlorophyll, the plant converts that sunlight into",
      score_cosine_similarity: 0.9308958623325689,
      score_euclidean_distance: 0.37116271429344766
    }
  ],
  cohere: [
    {
      prediction: "Photosynthesis is the process by which plants use sunlight, water, and carbon dioxide to create oxygen and energy in the form of sugar. Chlorophyll, a chemical in the plant's chloroplasts, absorbs sunlight, while water and carbon dioxide enter",
      score_cosine_similarity: 0.9472055942034299,
      score_euclidean_distance: 0.3227964066245007
    }
  ],