-
### System Info
It seems like executor API ignores `prompt_vocab_size` argument and passes `max_prompt_embedding_table_size` to trt engine instead.
I observe such behaviour using either 0.10.0 p…
-
import torch
from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, AutoModelForSeq2Se…
-
### Request for Assistance: Implementing AI Response Function with History Management
#### Objective
I am working on a function that should take a `String` parameter (prompt) and return a `String`…
-
Using the v0.8 version of [ChatQnA example](https://github.com/opea-project/GenAIExamples/blob/v0.8/ChatQnA/docker/gaudi/compose.yaml), the tgi service fails with heath test.
Environment:
- OS: ub…
-
I use this code to load the model:
```
model = 'WizardLM/WizardCoder-15B-V1.0'
def load_model(model = model):
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausal…
-
-
# Bug Report
Iam referring to [https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/language_model/llama/smooth_quant](https://github.com/microsoft/onnxruntime-inference…
-
The tutoriels for LLM topic generation use textgeneration.py or openai, thouse classes have this function to insert topics and documents into a custom prompt.
def _create_prompt(self, docs, topic, …
-
first of all congrats on your accomplishments!
but I must be doing something wrong but I can't get it to work,
I want to install it in my textgenwebui environment but I get this error:
`C:\tex…
-
Hello, thank you for providing the code and checkpoints. I would like to generate stories similar to those inferred in the paper, but the results I am getting are as follows. Could you please share th…