Open spinagon opened 3 months ago
There is no handling of long inputs, so small context models will fail. Longer context models (128k), work well, but for small context models chunking and aggregation are yet to be implemented.
Today, v0.1.12
was released with ChunksTabularExtractor
used by default. It allows the processing of pages with smaller context models by setting the chunk_size
and token_counter
arguments in Parsera
.
For details, check the documentation: https://docs.parsera.org/features/extractors/#chunks-tabular-extractor
When running with a llama.cpp model (
from langchain_community.llms import LlamaCpp
) I getValueError: Requested tokens (113654) exceed context window of 4096
I'm not sure what happens with other backends, they probably just trim everything that doesn't fit into context.