Cannot perform inference, be it unconditional. input-file or interactive

[srivassid]

Describe the bug I tried multiple ways of inference, but none of them work. I always get the same error. I am running it in a docker container.

Error

Context prompt >>> /home/mchorse/gpt-neox/megatron/text_generation_utils.py:163: UserWarning: The torch.cuda.*DtypeTensor constructors are no longer recommended. It's best to use methods such as torch.tensor(data, dtype=*, device='cuda') to create tensors. (Triggered internally at /opt/pytorch/pytorch/torch/csrc/tensor/python_tensor.cpp:83.) terminate_runs_tensor = torch.cuda.LongTensor([terminate_runs]) /home/mchorse/gpt-neox/megatron/text_generation_utils.py:163: UserWarning: The torch.cuda.*DtypeTensor constructors are no longer recommended. It's best to use methods such as torch.tensor(data, dtype=*, device='cuda') to create tensors. (Triggered internally at /opt/pytorch/pytorch/torch/csrc/tensor/python_tensor.cpp:83.) terminate_runs_tensor = torch.cuda.LongTensor([terminate_runs]) Traceback (most recent call last): File "/home/mchorse/gpt-neox/generate.py", line 93, in <module> main() File "/home/mchorse/gpt-neox/generate.py", line 75, in main generate_samples_interactive( File "/home/mchorse/gpt-neox/megatron/text_generation_utils.py", line 777, in generate_samples_interactive for ( File "/home/mchorse/gpt-neox/megatron/text_generation_utils.py", line 319, in stream_tokens logits[:, -1].view(batch_size, -1).contiguous() TypeError: tuple indices must be integers or slices, not tuple Traceback (most recent call last): File "/home/mchorse/gpt-neox/generate.py", line 93, in <module> main() File "/home/mchorse/gpt-neox/generate.py", line 75, in main generate_samples_interactive( File "/home/mchorse/gpt-neox/megatron/text_generation_utils.py", line 777, in generate_samples_interactive for ( File "/home/mchorse/gpt-neox/megatron/text_generation_utils.py", line 319, in stream_tokens logits[:, -1].view(batch_size, -1).contiguous() TypeError: tuple indices must be integers or slices, not tuple

To Reproduce

Command is python ./deepy.py generate.py -d configs 125M.yml local_setup.yml text_generation.yml

Expected behavior

The interactive window should allow me to talk to the model, or it should generate text based on file-based prompts.

Environment (please complete the following information):

3x3090

i5 13400f CUDA 12.2

[srivassid]

Used this branch and it is working fine now. Thanks

https://github.com/segyges/gpt-neox/tree/python-version-update

[iPRET]

Had the same issue, I just rewrote the line it crashed on in /megatron/text_generation_utils.py to logits[0][:, -1].view(batch_size, -1).contiguous() Started working