Summary

Describe the bug With transformers==4.37.2 installed, I am getting this error:

File ~/workspace/mlops-talk-llm-kg/venv/lib/python3.11/site-packages/transformers/generation/logits_process.py:1235, in PrefixConstrainedLogitsProcessor.__call__(self, input_ids, scores)
   1233         prefix_allowed_tokens = self._prefix_allowed_tokens_fn(batch_id, sent)
   1234         if len(prefix_allowed_tokens) == 0:
-> 1235             raise ValueError(
   1236                 f"`prefix_allowed_tokens_fn` returned an empty list for batch ID {batch_id}."
   1237                 f"This means that the constraint is unsatisfiable. Please check your implementation"
   1238                 f"of `prefix_allowed_tokens_fn` "
   1239             )
   1240         mask[batch_id * self._num_beams + beam_id, prefix_allowed_tokens] = 0
   1242 return scores + mask

ValueError: `prefix_allowed_tokens_fn` returned an empty list for batch ID 0.This means that the constraint is unsatisfiable. Please check your implementationof `prefix_allowed_tokens_fn`

If I roll back to the version of transformers in the Readme's Google Colabs, transformers==4.35.2, I don't have any issues.

To Reproduce

import spacy
from zshot import PipelineConfig, displacy
from zshot.linker import LinkerSMXM
from zshot.utils.data_models import Entity, Relation
from zshot.relation_extractor import RelationsExtractorZSRC
from zshot.mentions_extractor import MentionsExtractorSpacy
from zshot.linker import LinkerRegen

nlp = spacy.load('en_core_web_sm')

# zero shot definition of entities
nlp_config = PipelineConfig(
    mentions_extractor=MentionsExtractorSpacy(),
    linker=LinkerRegen(),
    entities=[
        Entity(name='Paris',
               description='Paris is located in northern central France, in a north-bending arc of the river Seine'),
        Entity(name='IBM',
               description='International Business Machines Corporation (IBM) is an American multinational technology corporation headquartered in Armonk, New York'),
        Entity(name='New York', description='New York is a city in U.S. state'),
        Entity(name='Florida', description='southeasternmost U.S. state'),
        Entity(name='American',
              description='American, something of, from, or related to the United States of America, commonly known as the United States or America'),
        Entity(name='Chemical formula',
               description='In chemistry, a chemical formula is a way of presenting information about the chemical proportions of atoms that constitute a particular chemical compound or molecul'),
        Entity(name='Acetamide',
               description='Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent.'),
        Entity(name='Armonk',
               description='Armonk is a hamlet and census-designated place (CDP) in the town of North Castle, located in Westchester County, New York, United States.'),
        Entity(name='Acetic Acid',
               description='Acetic acid, systematically named ethanoic acid, is an acidic, colourless liquid and organic compound with the chemical formula CH3COOH'),
        Entity(name='Industrial solvent',
               description='Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent.'),
    ]
)
nlp.add_pipe('zshot', config=nlp_config, last=True)

text = 'International Business Machines Corporation (IBM) is an American multinational technology corporation' \
        ' headquartered in Armonk, New York, with operations in over 171 countries.'

doc = nlp(text)
displacy.render(doc, style="ent", jupyter=True)

Expected behavior This should work without any issue

IBM / zshot

[Bug] ValueError: `prefix_allowed_tokens_fn` with new version of `transformers` package #73

Summary