python BERT_MPS_PYTORCH/transformer_sequence_classifier.py --device mps
Some weights of the model checkpoint at bert-base-cased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight']
This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
INFO:root:Input tensors size:
INFO:root: input_ids: torch.Size([32, 128])
INFO:root: attention_mask: torch.Size([32, 128])
INFO:root: * labels: torch.Size([32])
Testing...: 0%| | 0/100 [00:00<?, ?it/s]-:8:10: error: Rank of data array must equal number of outer dimensions in indices array + rank of slice to update, 1 != 1 + 1
-:8:10: note: see current operation: %5 = "mps.scatter_nd"(%0, %arg0, %4) {batch_dims = 0 : ui32, mode = 0 : i32} : (tensor<1xf32>, tensor<1x128x768xf32>, tensor<1x128x1xi64>) -> tensor<1xf32>
/AppleInternal/Library/BuildRoots/b6051351-c030-11ec-96e9-3e7866fcf3a1/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:1267: failed assertion `Error: MLIR pass manager failed'
[1] 273 abort python BERT_MPS_PYTORCH/transformer_sequence_classifier.py --device mps
/Users/jordanharris/.pyenv/versions/3.9.12/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Not exactly sure how to read this error.
python BERT_MPS_PYTORCH/transformer_sequence_classifier.py --device mps Some weights of the model checkpoint at bert-base-cased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight']