search

vikhyat / moondream

tiny vision language model
https://moondream.ai
Apache License 2.0
4.87k stars 431 forks source link

performance regression on MPS #117

Open vikhyat opened 1 month ago

vikhyat commented 1 month ago

Appears to be caused by this:

/Users/vikhyat/Coding/moondream/.venv/lib/python3.12/site-packages/transformers/generation/utils.py:1513: UserWarning: The operator 'aten::isin.Tensor_Tensor_out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:13.)
  if eos_token_id is not None and torch.isin(elements=eos_token_id, test_elements=pad_token_id).any():
vikhyat commented 1 month ago

Looks like this was fixed here: https://github.com/huggingface/transformers/pull/31695