Open davemssavage opened 7 months ago
Rather than large scale changes it might be better in the short term to do a type check on the constructor for Compel to assert that the tokenizers are CLIPTokenizers as per the method signature - as python doesn't actually enforce this at runtime hence the confusing down stream errors. Perhaps an update to the README.md file as well would be useful as it seems to suggest compel will work with any Tokenizer but that is evidently not the case.
thanks for the comments, yes, type checking would probably be a good idea.
This issue likely affects any tokenizer that does not define bos_token of which T5Tokenizer is an example:
Steps to reproduce use PixArt-alpha/PixArt-XL-2-1024-MS that uses T5Tokenizer:
Debug of the code suggests the problem lies here and here a simple hacky fix is to replace:
[self.tokenizer.bos_token_id]
with([self.tokenizer.pad_token_id] if self.tokenizer.bos_token_id is None else [self.tokenizer.bos_token_id])
However I suspect this logic will only work for the T5 Tokenizer which has the following note in the docs
Unfortunately it still does not work with PixelArt-Alpha as that pipeline expects an attention mask when passing in embeds directly and that will also need wider changes to Compel to support