-
OpenAI 3.5 and 4o are included.
However, many open-source alternative LLMs exist too.
**TODO: decide which open-source alternatives to use.**
-
reference: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/quantization-in-TRT-LLM.md#performance
![image](https://github.com/user-attachments/assets/1bb20225-3eb2-4641-b5ba-f027…
-
from line 107 to 111
start_idx = 0
for i, doc_ids in enumerate(examples['doc_ids']):
end_idx = start_idx + len(doc_ids)
examples['doc_scores'][i] = llm_scores[start_idx:end_idx]
sta…
-
### System Info
- CPU architecture: x86_64
- Host memory: 256GB
- GPU
+ Name: NVIDIA A30
+ Memory: 24GB
- Libraries
+ TensorRT-LLM: v0.11.0
+ TensorRT: 10.1.0
+ CUDA: 12.6
+ NVID…
-
Hi,
I am starting to play with Baml, and I am very excited about it.
## Problem
I stumble upon a strange case, where a test in PromptFilddle is failing, but the LLM output seems correct.
Ind…
-
### What happened?
# Issue
After a few recent updates, llama.cpp crashes with a segmentation fault (core dumped) error.
This happens with any large model.
**It didn't happen before with the same…
-
### Describe the bug
When attempting to run "interpreter --local" and choosing jan.ai as the llm provider, the model choice function crashes interpreter.
LM_Studio runs as expected. (I'm assumi…
-
gpt-j from ipex-llm/python/llm/example/GPU/HF-Transformers-AutoModels/Model/gpt-j
$ python generate.py
2024-03-31 17:42:23,819 - INFO - intel_extension_for_pytorch auto imported
2024-03-31 17:42:…
-
Post questions here for this week's exemplary readings:
3. Kim, J., & Lee, B. “[AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction](https://arxiv.org/abs/2305…
lkcao updated
6 months ago
-
https://github.com/uclaml/SPPO/blob/e524519cc87e9e48cd4da30588f7aa566638df4c/scripts/compute_prob.py#L39
From my understanding of the code, the score list here is the output from the `blender.rank(…
namdw updated
2 weeks ago