-
Despite `do_sample` being set to `False`, we are occasionally (1-2% of the time) seeing TGI running Llama-7B models on INF2-Sagemaker return different outputs, despite being passed identical inputs. I…
-
You mentioned in the readme that max_length_generation=512 is enough for tasks like HumanEval and MBPP, but when I tested phi-1.5 and deepseek-coder-1.3b-base on the mbpp task, the following problems …
-
-
Hi,
I am trying to connect llama-2-7b.ggmlv3.q4_K_S.bin from DSPy code and i am not able to find right way to do this. Can someone please explain or guide me to a document on do this? Btw, i am able …
-
Hello! I've been attempting to use together.ai models for this project, and I even checked out the https://github.com/princeton-nlp/SWE-agent/blob/main/sweagent/agent/models.py, but I can't seem to ge…
-
I met this problem.
outputs = llm.generate(prompts, sampling_params)
File "/home/ma-user/anaconda3/envs/py39/lib/python3.9/site-packages/vllm/entrypoints/llm.py", line 165, in generate
re…
-
I noticed that this repository has already added its own dataset for leetcode, but it is not yet enabled to be one of the "available" datasets.
Command:
python3 automodel.py --name codellama/Cod…
-
I did not really measure, whether it's infinite (I did hit Ctrl+C before infinity), but it suddenly started to repeat itself:
tvali@PC366:~$ ollama run codellama:7b
>>> Is LDM a Deep Learning mode…
-
I updated Ollama from 0.1.16 to 0.1.18 and encountered the issue.
I am using python to use LLM models with Ollama and Langchain on Linux server(4 x A100 GPU).
There are 5,000 prompts to ask and get…
-
Get models work
```json
{
"data": [
{
"id": "codellama/CodeLlama-70b-Instruct-hf"
},
{
"id": "codellama/CodeLlama-34b-Instruct-hf"
},
{
"id": "jondur…