-
OpenAI 3.5 and 4o are included.
However, many open-source alternative LLMs exist too.
**TODO: decide which open-source alternatives to use.**
-
reference: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/quantization-in-TRT-LLM.md#performance
![image](https://github.com/user-attachments/assets/1bb20225-3eb2-4641-b5ba-f027…
-
De plus en plus, je pense qu'on a besoin de tester les releases. Je me suis rendu compte trop tardivement des problèmes importants sur l'itinéraire. Des tests complets, qui simulent des utilisateurs d…
-
from line 107 to 111
start_idx = 0
for i, doc_ids in enumerate(examples['doc_ids']):
end_idx = start_idx + len(doc_ids)
examples['doc_scores'][i] = llm_scores[start_idx:end_idx]
sta…
-
### System Info
- CPU architecture: x86_64
- Host memory: 256GB
- GPU
+ Name: NVIDIA A30
+ Memory: 24GB
- Libraries
+ TensorRT-LLM: v0.11.0
+ TensorRT: 10.1.0
+ CUDA: 12.6
+ NVID…
-
### Common description
I have encountered an issue where Baml parses the response from the LLM incorrectly. Specifically, it seems that one block type is being incorrectly converted into another bloc…
-
Hi,
I am starting to play with Baml, and I am very excited about it.
## Problem
I stumble upon a strange case, where a test in PromptFilddle is failing, but the LLM output seems correct.
Ind…
-
### Describe the bug
When attempting to run "interpreter --local" and choosing jan.ai as the llm provider, the model choice function crashes interpreter.
LM_Studio runs as expected. (I'm assumi…
-
### What is the issue?
Your branch is up to date with 'origin/minicpm-v2.5'.
Already on 'minicpm-v2.5'
Submodule path '../llama.cpp': checked out 'd8974b8ea61e1268a4cad27f4f6e2cde3c5d1370'
Checkin…
-
gpt-j from ipex-llm/python/llm/example/GPU/HF-Transformers-AutoModels/Model/gpt-j
$ python generate.py
2024-03-31 17:42:23,819 - INFO - intel_extension_for_pytorch auto imported
2024-03-31 17:42:…