TypeError: __new__() got multiple values for argument 'cls'

[phosseini]

I'm trying to run the pretraining code and I get the following error:

Traceback (most recent call last):
  File "train.py", line 396, in <module>
    main(args)
  File "train.py", line 47, in main
    load_dataset_splits(task, ['train'] + args.valid_subset.split(','))
  File "train.py", line 371, in load_dataset_splits
    task.load_dataset(split, combine=True)
  File "/home/ubuntu/projects/SpanBERT/pretraining/fairseq/tasks/span_bert.py", line 180, in load_dataset
    tag_map=tag_map
TypeError: __new__() got multiple values for argument 'cls'

The command I'm running (for now I intentionally changed the distributed_port and distributed_world_size to avoid the distributed or multiprocessing mode):

python train.py ../data/destination/ --total-num-update 2400000 --max-update 2400000 --save-interval 1 --arch cased_bert_pair_large --task span_bert --optimizer adam --lr-scheduler polynomial_decay --lr 0.0001 --min-lr 1e-09 --criterion span_bert_loss --max-tokens 4096 --tokens-per-sample 512 --weight-decay 0.01 --skip-invalid-size-inputs-valid-test --log-format json --log-interval 2000 --save-interval-updates 50000 --keep-interval-updates 50000 --update-freq 1 --seed 1 --save-dir ../data/checkpoint_dir --fp16 --warmup-updates 10000 --schemes [\"pair_span\"] --distributed-port 0 --distributed-world-size 1 --span-lower 1 --span-upper 10 --validate-interval 1 --clip-norm 1.0 --geometric-p 0.2 --adam-eps 1e-8 --short-seq-prob 0.0 --replacement-method span --clamp-attention --no-nsp --pair-loss-weight 1.0 --max-pair-targets 15 --pair-positional-embedding-size 200 --endpoints external

I've successfully preprocessed and tokenized the corpus, so I have no problem with that. I have the following files in the ../data/destination/:

• dict.txt
• train.bin
• train.idx
• valid.bin
• valid.idx

Even though I know what possibly the TypeError: __new__() got multiple values for argument could mean, I have no idea what causes such an error here. Any insight or cue is appreciated!

@mandarjoshi90 P.S. I know that CPU is not supported, but just to test the rest of the code, I temporarily removed the check for availability of the CPU and I did not get the error I mentioned above and could load the data successfully. So I wonder if this error I'm getting has anything to do with running the code on GPU or any GPU-related packages?

[phosseini]

Apparently, this error was caused by the torch version incompatibility. I installed torch==1.2.0 and this error is resolved.

[jakeywu]

https://stackoverflow.com/questions/21764770/typeerror-got-multiple-values-for-argument

[PhSe-coder]

This may be because the name of the keyword parameter cls in the block_cls class and the formal parameter name cls of the __new__ magic method are duplicated, you can delete the keyword parameter statement in the block_cls from line 170-173 just like this:

with data_utils.numpy_seed(self.seed + k):
    loaded_datasets.append(
        block_cls(
        tokens,
        ds.sizes,
        self.args.tokens_per_sample,
        self.dictionary.pad(),
        self.dictionary.cls(),
        self.dictionary.mask(),
        self.dictionary.sep(),
        break_mode=self.args.break_mode,
        short_seq_prob=self.short_seq_prob,
        tag_map=tag_map

        ))