RuntimeError: selected index k out of range

[isaacdecoded]

When trying the "Multilingual suggestions" example an error occurs:

/usr/local/lib/python3.6/dist-packages/checklist/text_generation.py in unmask(self, text_with_mask, beam_size, candidates) 180 else: 181 if forbid: --> 182 v, top_preds = torch.topk(outputs[i, masked[size], self.with_space], beam_size + 10) 183 top_preds = self.with_space[top_preds] 184 else:

RuntimeError: selected index k out of range

[marcotcr]

It works for me. Which example are you trying to run? Portuguese or chinese? Can you please paste the snippet, as well as your version of python, checklist, transformers, torch? You can run python --version and pip show checklist transformers torch

[isaacdecoded]

Thanks for your reply, Marco. I'm trying with Portuguese, the code I'm running is:

import checklist
from checklist.editor import Editor
import numpy as np
# in Portuguese
editor = Editor(language='portuguese')
ret = editor.template('O João é um {mask}.',)
ret.data[:3]

Versions: python v3.6 checklist v0.0.9 torch v1.6.0+cu101 Not using transformers

Maybe there is a step I'm not doing well or missing?

Thanks.

[marcotcr]

Hm, transformers is a dependency of checklist and definitely needed for suggestions, did you by any chance uninstall it manually?

[isaacdecoded]

Oh, you're right, transformers is present as a dependency. The version I have now is 3.1.0. By the way, I'm trying it on a docker container. Building with following install commands:

pip install checklist pip install torch

[marcotcr]

I'm trying to replicate, but can't (even with the same configuration as you). I'm assuming you're using an Ubuntu container? And can you run the non-multilingual suggestion examples?

[isaacdecoded]

Yes, it only fails when trying to use suggestions through {mask} tag, all the other features works good.

I'm testing on a Google Colab instance as well and the same errors occurs:

https://colab.research.google.com/drive/1IBlZKUYBgjbUUEyFGKjgsPJ5OLPlUB1Y?usp=sharing

And the Dockerfile I'm building:

FROM python:3

ADD test/checklist/test_rules.py /

ADD etc/checklist_requirements.txt /requirements.txt

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt && \
    jupyter nbextension install --py --sys-prefix checklist.viewer && \
    jupyter nbextension enable --py --sys-prefix checklist.viewer

CMD [ "python", "./test_rules.py" ]

test_rules.py just have the Portuguese multilingual suggestion example.

[marcotcr]

Thanks a lot, this is really helpful. There was a bug in our code interacting with the new transformers tokenizers, fixed in 6159631ba0487d4410be8479bb4dff137054dd7d and in checklist 0.0.10. Please let me know if it doesn't work after you update (it should)

[isaacdecoded]

Working now!

Thanks very much to you and your help. And of course, your contribution.