Getting error 'RuntimeError: expected scalar type Long but found Int' while running the starting code

[Nim248]

Bug description

I am getting the following error

RuntimeError: expected scalar type Long but found Int

related to the line

indices_labeled = initialize_active_learner(active_learner, train.y)

in the code provided here

https://github.com/webis-de/small-text/blob/v1.1.1/examples/notebooks/02-active-learning-with-stopping-criteria.ipynb

I am using the latest version.

Python version: 3.8.8 small-text version: 1.1.1 torch version (if applicable): 1.13.0+cpu

Full error:

RuntimeError Traceback (most recent call last)

in 28 29 active_learner = PoolBasedActiveLearner(clf_factory, query_strategy, train) ---> 30 indices_labeled = initialize_active_learner(active_learner, train.y) 31 in initialize_active_learner(active_learner, y_train) 12 13 indices_initial = random_initialization_balanced(y_train, n_samples=20) ---> 14 active_learner.initialize_data(indices_initial, y_train[indices_initial]) 15 16 return indices_initial ~\Anaconda3\lib\site-packages\small_text\active_learner.py in initialize_data(self, indices_initial, y_initial, indices_ignored, indices_validation, retrain) 149 150 if retrain: --> 151 self._retrain(indices_validation=indices_validation) 152 153 def query(self, num_samples=10, representation=None, query_strategy_kwargs=dict()): ~\Anaconda3\lib\site-packages\small_text\active_learner.py in _retrain(self, indices_validation) 388 389 if indices_validation is None: --> 390 self._clf.fit(dataset) 391 else: 392 indices = np.arange(self.indices_labeled.shape[0]) ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in fit(self, train_set, validation_set, weights, early_stopping, model_selection, optimizer, scheduler) 366 use_sample_weights=weights is not None) 367 --> 368 return self._fit_main(sub_train, sub_valid, sub_train_weights, early_stopping, 369 model_selection, fit_optimizer, fit_scheduler) 370 ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _fit_main(self, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler) 389 390 with tempfile.TemporaryDirectory(dir=get_tmp_dir_base()) as tmp_dir: --> 391 self._train(sub_train, sub_valid, weights, early_stopping, model_selection, 392 optimizer, scheduler, tmp_dir) 393 self._perform_model_selection(optimizer, model_selection) ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train(self, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir) 435 start_time = datetime.datetime.now() 436 --> 437 train_acc, train_loss, valid_acc, valid_loss, stop = self._train_loop_epoch(epoch, 438 sub_train, 439 sub_valid, ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_loop_epoch(self, num_epoch, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir) 471 validate_every = None 472 --> 473 train_loss, train_acc, valid_loss, valid_acc, stop = self._train_loop_process_batches( 474 num_epoch, 475 sub_train, ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_loop_process_batches(self, num_epoch, sub_train_, sub_valid_, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir, validate_every) 505 for i, (x, masks, cls, weight, *_) in enumerate(train_iter): 506 if not stop: --> 507 loss, acc = self._train_single_batch(x, masks, cls, weight, optimizer) 508 scheduler.step() 509 ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_single_batch(self, x, masks, cls, weight, optimizer) 561 outputs = self.model(x, attention_mask=masks) 562 --> 563 logits, loss = self._compute_loss(cls, outputs) 564 loss = loss * weight 565 loss = loss.mean() ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _compute_loss(self, cls, outputs) 585 logits = outputs.logits.view(-1, self.num_classes) 586 target = cls --> 587 loss = self.criterion(logits, target) 588 589 return logits, loss ~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs) 1188 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks 1189 or _global_forward_hooks or _global_forward_pre_hooks): -> 1190 return forward_call(*input, **kwargs) 1191 # Do not call functions when jit is used 1192 full_backward_hooks, non_full_backward_hooks = [], [] ~\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target) 1172 1173 def forward(self, input: Tensor, target: Tensor) -> Tensor: -> 1174 return F.cross_entropy(input, target, weight=self.weight, 1175 ignore_index=self.ignore_index, reduction=self.reduction, 1176 label_smoothing=self.label_smoothing) ~\Anaconda3\lib\site-packages\torch\nn\functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing) 3024 if size_average is not None or reduce is not None: 3025 reduction = _Reduction.legacy_get_string(size_average, reduce) -> 3026 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing) 3027 3028 RuntimeError: expected scalar type Long but found Int

[chschroeder]

Hi @Nim248, thanks for reporting this.

Unfortunately, I cannot reproduce your problem. I have tried both a colab instance and a local setup (including several configurations of pytorch/transformer versions) for testing but never encountered this error.

Did you modify the original code?

[Nim248]

The only change I made is here (changed cuda to cpu). I am using "small-text" without "[transformers]" on a Jupyter notebook.

clf_factory = TransformerBasedClassificationFactory(transformer_model, num_classes, kwargs=dict({ 'device': 'cpu', 'mini_batch_size': 32, 'class_weight': 'balanced' }))

[chschroeder]

The only change I made is here (changed cuda to cpu).

Strange, still no error for me. Can you try again with a fresh notebook (use the link in the README.md).

Besides, are you sure that you want to use this without a GPU? The training process takes so much longer on a CPU. A GPU from Colab is sufficient for many models.

I am using "small-text" without "[transformers]" on a Jupyter notebook.

This is likely not the case, probably even impossible. The error you posted shows the transformer classifier in the stacktrace. If you tried to use small-text transformers functionality without having transformers installed there will be errors much earlier (e.g., when you try to prepare the data).

Maybe you installed it during your first try. The "transformers" seems to be still present here. This is why I recommended to start with a clean environment above.

[Nim248]

There does seem to be a mismatch in libraries. I have installed cuda and pytorch but this piece of code still gives me another error:

RuntimeError Traceback (most recent call last)

in 28 29 active_learner = PoolBasedActiveLearner(clf_factory, query_strategy, train) ---> 30 indices_labeled = initialize_active_learner(active_learner, train.y) 31 in initialize_active_learner(active_learner, y_train) 12 13 indices_initial = random_initialization_balanced(y_train, n_samples=20) ---> 14 active_learner.initialize_data(indices_initial, y_train[indices_initial]) 15 16 return indices_initial ~\Anaconda3\lib\site-packages\small_text\active_learner.py in initialize_data(self, indices_initial, y_initial, indices_ignored, indices_validation, retrain) 149 150 if retrain: --> 151 self._retrain(indices_validation=indices_validation) 152 153 def query(self, num_samples=10, representation=None, query_strategy_kwargs=dict()): ~\Anaconda3\lib\site-packages\small_text\active_learner.py in _retrain(self, indices_validation) 388 389 if indices_validation is None: --> 390 self._clf.fit(dataset) 391 else: 392 indices = np.arange(self.indices_labeled.shape[0]) ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in fit(self, train_set, validation_set, weights, early_stopping, model_selection, optimizer, scheduler) 366 use_sample_weights=weights is not None) 367 --> 368 return self._fit_main(sub_train, sub_valid, sub_train_weights, early_stopping, 369 model_selection, fit_optimizer, fit_scheduler) 370 ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _fit_main(self, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler) 389 390 with tempfile.TemporaryDirectory(dir=get_tmp_dir_base()) as tmp_dir: --> 391 self._train(sub_train, sub_valid, weights, early_stopping, model_selection, 392 optimizer, scheduler, tmp_dir) 393 self._perform_model_selection(optimizer, model_selection) ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train(self, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir) 435 start_time = datetime.datetime.now() 436 --> 437 train_acc, train_loss, valid_acc, valid_loss, stop = self._train_loop_epoch(epoch, 438 sub_train, 439 sub_valid, ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_loop_epoch(self, num_epoch, sub_train, sub_valid, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir) 471 validate_every = None 472 --> 473 train_loss, train_acc, valid_loss, valid_acc, stop = self._train_loop_process_batches( 474 num_epoch, 475 sub_train, ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_loop_process_batches(self, num_epoch, sub_train_, sub_valid_, weights, early_stopping, model_selection, optimizer, scheduler, tmp_dir, validate_every) 505 for i, (x, masks, cls, weight, *_) in enumerate(train_iter): 506 if not stop: --> 507 loss, acc = self._train_single_batch(x, masks, cls, weight, optimizer) 508 scheduler.step() 509 ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _train_single_batch(self, x, masks, cls, weight, optimizer) 561 outputs = self.model(x, attention_mask=masks) 562 --> 563 logits, loss = self._compute_loss(cls, outputs) 564 loss = loss * weight 565 loss = loss.mean() ~\Anaconda3\lib\site-packages\small_text\integrations\transformers\classifiers\classification.py in _compute_loss(self, cls, outputs) 581 if self.num_classes == 2: 582 logits = outputs.logits --> 583 target = F.one_hot(cls, 2).float() 584 else: 585 logits = outputs.logits.view(-1, self.num_classes) RuntimeError: one_hot is only applicable to index tensor. However, the colab note seems to be running fine. Closing this issue since it is potentially due to misconfiguration.

[chschroeder]

Thank you again for the feedback and for updating the issue.

The problem could be due to a version mismatch, but the resulting error is not helpful. I will investigate this further.