AttributeError: 'tuple' object has no attribute 'cls_token'

[arachny]

I have been trying to get this to work for several days now and keep on getting errors every time. I tried building the container image on my mac and on an AWS p3.8xlarge instance, but failed each time.

So this is the error I am getting if I just build the docker image as is (latest version) and then try it on the toxic-comments dataset, as is described exactly in the blog.

UnexpectedStatusException: Error for Training job toxic-comments-2020-04-24-20-21-27-316: Failed. Reason: AlgorithmError: Exception during training: 'tuple' object has no attribute 'cls_token'
Traceback (most recent call last):
  File "/opt/ml/code/train", line 157, in train
    logger=logger,
  File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 421, in __init__
    train_examples, "train", no_cache=self.no_cache
  File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 542, in get_dataset_from_examples
    cls_token=self.tokenizer.cls_token,
AttributeError: 'tuple' object has no attribute 'cls_token'

It appears that this error relates to AutoTokenizer, so I even tried to replace that to how the old version had it (RobertaTokenizer, etc.), but i was then getting a different error: "Caught StopIteration in replica 0 on device 0."

I tried to install fast-bert and apex directly into a notebook (with GPU) and then run the training and prediction code locally (i.e. not in the container via sagemaker) and it all runs fine!

Any ideas on what can be going wrong with that container?

[RajatHanda]

Any update on this issue. It will be helpful if version information can be added to requirements.txt

[ocryan9068]

I'm having the same issue:

`06/24/2020 13:31:35 - INFO - transformers.tokenization_utils - loading file None 06/24/2020 13:31:35 - INFO - root - Number of GPUs: 4 06/24/2020 13:31:35 - INFO - root - label columns: ['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate'] Exception during training: 'tuple' object has no attribute 'cls_token' Traceback (most recent call last): File "/opt/ml/code/train", line 157, in train logger=logger, File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 424, in init train_examples, "train", no_cache=self.no_cache File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 550, in get_dataset_from_examples cls_token=self.tokenizer.cls_token, AttributeError: 'tuple' object has no attribute 'cls_token'

---

UnexpectedStatusException Traceback (most recent call last)

in ----> 1 estimator.fit(s3_input) /opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config) 495 self.jobs.append(self.latest_training_job) 496 if wait: --> 497 self.latest_training_job.wait(logs=logs) 498 499 def _compilation_job_name(self): /opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in wait(self, logs) 1107 # If logs are requested, call logs_for_jobs. 1108 if logs != "None": -> 1109 self.sagemaker_session.logs_for_job(self.job_name, wait=True, log_type=logs) 1110 else: 1111 self.sagemaker_session.wait_for_job(self.job_name) /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in logs_for_job(self, job_name, wait, poll, log_type) 3066 3067 if wait: -> 3068 self._check_job_status(job_name, description, "TrainingJobStatus") 3069 if dot: 3070 print() /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in _check_job_status(self, job, desc, status_key_name) 2660 ), 2661 allowed_statuses=["Completed", "Stopped"], -> 2662 actual_status=status, 2663 ) 2664 UnexpectedStatusException: Error for Training job toxic-comments-2020-06-24-13-26-42-334: Failed. Reason: AlgorithmError: Exception during training: 'tuple' object has no attribute 'cls_token' Traceback (most recent call last): File "/opt/ml/code/train", line 157, in train logger=logger, File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 424, in __init__ train_examples, "train", no_cache=self.no_cache File "/opt/conda/lib/python3.7/site-packages/fast_bert/data_cls.py", line 550, in get_dataset_from_examples cls_token=self.tokenizer.cls_token, AttributeError: 'tuple' object has no attribute 'cls_token'`

[ocryan9068]

Pull request created to fix this: #239