Add Generator module that supports log probs and tokens.

Marker-Inc-Korea / AutoRAG

RAG AutoML Tool - Find optimal RAG pipeline for your own data.

Apache License 2.0

1.12k stars 97 forks source link

Add Generator module that supports log probs and tokens. #63

Open vkehfdl1 opened 5 months ago

vkehfdl1 commented 5 months ago

Generator Node designed to return log probs and tokens as default, but as we can't get these informations at llama index, we had to make log probs llm module for this feature.

vkehfdl1 commented 3 months ago

Now vllm supports proper logprobs and output tokens.