-
Hi,
It fails to create context from binary for the 4 QNN Context Bin files (llama_v2_7b_chat_quantized_TokenGenerator_1_Quantized.bin, llama_v2_7b_chat_quantized_TokenGenerator_2_Quantized.bin, llama…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [X] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
When I lower this toy example to QNN backend, the linear operator cause an error.
```
class MutableStateModule(torch.nn.Module):
def __init__(self):
super().__init__()
…
-
### Issue description
When I use node-llama-cpp to run inference, cloudrun fails with a 503 error.
### Expected Behavior
Run inference in cloudrun without any issues.
### Actual Behavior
…
-
Any ideas, why it can be slow? For example I'm using KoboldCPP with the same Mistral model and it answers immediately in realtime, almost like ChatGPT (I have RTX 4090). It also starts in like 15 seco…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hello there, I have an use case for my company of which I set up a naive RAG + reranker …
-
I get the following error stack when running the jupyter notebook cell with the line
`graph_documents = llm_transformer.convert_to_graph_documents(documents)`
The stack is as follows
`{
"na…
-
### What happened?
Native API failed. Native API returns: -999 (Unknown PI error) -999 (Unknown PI error)
Exception caught at file:D:/a/llama.cpp/llama.cpp/ggml/src/ggml-sycl.cpp, line:4207, func:…
-
### Bug Description
from typing import Literal
from llama_index.llms.ollama import Ollama
from llama_index.core.indices.property_graph import SchemaLLMPathExtractor
from llama_index.vector_stores.…
-
Error occurred when executing Joy_caption:
`rope_scaling` must be a dictionary with two fields, `type` and `factor`, got {'factor': 8.0, 'high_freq_factor': 4.0, 'low_freq_factor': 1.0, 'original_m…