Closed eu9ene closed 1 week ago
It seems there are some OOMs for shortlist and unlike alignments, it's based on a proper sentence piece tokenization. There might be something else in the Python code and I'm investigating it now but in the meantime let's bump memory for it too.
It seems there are some OOMs for shortlist and unlike alignments, it's based on a proper sentence piece tokenization. There might be something else in the Python code and I'm investigating it now but in the meantime let's bump memory for it too.