search

Hannibal046 / xRAG

Source code for xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
88 stars 5 forks source link

stopping criteria bug #8

Open davidmrau opened 3 months ago

davidmrau commented 3 months ago

The stopping criteria only works for batch size = 1. I don't get an error, but no stopping criteria is applied at all for batch size > 1

Hannibal046 commented 3 months ago

Hi, can you try the transformers version specified in the docker file? I can not reproduce this error on my side.