RuntimeError: __init__() expected at most 3 argument(s) but received 5 argument(s)

[jeffhataws]

With latest 0.5 version and Neuron SDK 2.12, some tutorials like this https://github.com/aws-neuron/aws-neuron-samples/blob/master/torch-neuronx/transformers-neuronx/inference/gpt-j-6b-sampling.ipynb is hitting error "RuntimeError: init() expected at most 3 argument(s) but received 5 argument(s)":

Downloading (…)lve/main/config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 930/930 [00:00<00:00, 264kB/s]Downloading pytorch_model.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24.2G/24.2G [04:59<00:00, 80.9MB/s]
.....                                                                                                                                                                                           
Compiler status PASS                                                                                                                                                                            
....                                                                                                                                                                                            
Compiler status PASS                                                                                                                                                                            
....                                                                                                                                                                                            
Compiler status PASS                                                                                                                                                                            
....                                                                                                                                                                                            
Compiler status PASS                                                                                                                                                                            
Traceback (most recent call last):                                                                                                                                                              
  File "gptj.py", line 28, in <module>                                                                                                                                                          
    neuron_model.to_neuron()                                                                                                                                                                    
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/transformers_neuronx/gptj/model.py", line 72, in to_neuron                                                             
    self.program.setup(self.transformer.h, self.ln_lm_head)                                                                                                                                     
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/transformers_neuronx/program.py", line 102, in setup                                                                   
    kernel.load()                                                                                                                                                                               
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/transformers_neuronx/compiler.py", line 376, in load                                                                   
    self.model = torch.classes.neuron.ParallelModel(self.neff_bytes, self.tp_degree, self.g_start_device_id, self.g_device_count)                                                               
RuntimeError: __init__() expected at most 3 argument(s) but received 5 argument(s). Declaration: __init__(__torch__.torch.classes.neuron.ParallelModel _0, str _1, int _2) -> NoneType _0       

[awsilya]

Thanks for reporting. For this tutorial please install the older version of transformers-neuronx as GPT-J has not changed since then. pip install git+https://github.com/aws-neuron/transformers-neuronx.git@v0.4.0 transformers -U We will have a fix soon for this.

[jeffhataws]

The following change will also work, to install the 2.12 released transformers-neuronx pip wheel:

pip install transformers-neuronx transformers -U

Will update the example notebooks with this change.

[jeffhataws]

Tutorial has been update to install transformers-neuronx pip wheel instead of from github.

https://github.com/aws-neuron/aws-neuron-samples/blob/master/torch-neuronx/transformers-neuronx/inference/gpt-j-6b-sampling.ipynb