Error Training intel optimised version of disease-prediction

[krishnashed]

$ python run_training.py --logfile ../logs/intel.log --save_model_dir ../saved_models/intel --data_dir ../data/disease-prediction --intel Some weights of the model checkpoint at emilyalsentzer/Bio_ClinicalBERT were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.seq_relationship.bias', 'cls.predictions.bias']

• This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertForSequenceClassification were not initialized from the model checkpoint at emilyalsentzer/Bio_ClinicalBERT and are newly initialized: ['classifier.weight', 'classifier.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. /home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/intel_extension_for_pytorch/optim/_optimizer_utils.py:207: UserWarning: Does not suport fused step for <class 'torch.optim.adam.Adam'>, will use non-fused step warnings.warn("Does not suport fused step for " + str(type(optimizer)) + ", will use non-fused step") Epoch 1: 0%| | 0/133 [00:00<?, ?it/s] Traceback (most recent call last): File "run_training.py", line 221, in main(FLAGS) File "run_training.py", line 99, in main train( File "/home/ubuntu/referencekits/disease-prediction/src/utils/train.py", line 50, in train for , (batch, labels) in tqdm( File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/tqdm/std.py", line 1195, in iter for obj in iterable: File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in next data = self._next_data() File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 570, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/reference_kits/disease-prediction/src/utils/process_data.py", line 94, in getitem encoding = self.tokenizer( File "/home/ubuntu/anaconda3/envs/disease_pred_intel/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2489, in call raise ValueError( ValueError: text input must of type str (single example), List[str] (batch or single pretokenized example) or List[List[str]] (batch of pretokenized examples).

[devpramod-intel]

Hi @krishnashed, I am unable to reproduce this error on my end. Could you please print self.symptoms[idx] and it's type type(self.symptoms[idx]) on line 95 in file src/utils/process_data.py and share the result?

Could you also please provide outputs for conda list and

import torch.utils.collect_env as environ
print(environ.main())

[suhasdas]

I am getting same error too. Would you please provide solution for fixing this error. and I have tried in different laptop, still same issue.

[suhasdas]

here is the conda list:

packages in environment at /root/anaconda3/envs/disease_pred_stock:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_kmp_llvm conda-forge blas 2.120 mkl conda-forge blas-devel 3.9.0 20_linux64_mkl conda-forge brotli-python 1.1.0 py38h17151c0_1 conda-forge bzip2 1.0.8 hd590300_5 conda-forge ca-certificates 2023.11.17 hbcca054_0 conda-forge certifi 2023.11.17 pyhd8ed1ab_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge cpuonly 2.0 0 pytorch filelock 3.13.1 pypi_0 pypi fsspec 2023.12.2 pypi_0 pypi huggingface-hub 0.20.2 pypi_0 pypi icu 73.2 h59595ed_0 conda-forge idna 3.6 pyhd8ed1ab_0 conda-forge joblib 1.3.2 pyhd8ed1ab_0 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge libblas 3.9.0 20_linux64_mkl conda-forge libcblas 3.9.0 20_linux64_mkl conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 13.2.0 h807b86a_3 conda-forge libgfortran-ng 13.2.0 h69a702a_3 conda-forge libgfortran5 13.2.0 ha4646dd_3 conda-forge libhwloc 2.9.3 default_h554bfaf_1009 conda-forge libiconv 1.17 hd590300_2 conda-forge liblapack 3.9.0 20_linux64_mkl conda-forge liblapacke 3.9.0 20_linux64_mkl conda-forge libnsl 2.0.1 hd590300_0 conda-forge libsqlite 3.44.2 h2797004_0 conda-forge libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libuv 1.46.0 hd590300_0 conda-forge libxcrypt 4.4.36 hd590300_1 conda-forge libxml2 2.12.4 h232c23b_1 conda-forge libzlib 1.2.13 hd590300_5 conda-forge llvm-openmp 17.0.6 h4dfa4b3_0 conda-forge mkl 2023.2.0 h84fe81f_50496 conda-forge mkl-devel 2023.2.0 ha770c72_50496 conda-forge mkl-include 2023.2.0 h84fe81f_50496 conda-forge ncurses 6.4 h59595ed_2 conda-forge numpy 1.24.4 py38h59b608b_0 conda-forge openssl 3.2.0 hd590300_1 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 1.4.4 py38h47df419_0 conda-forge pip 23.3.2 pyhd8ed1ab_0 conda-forge platformdirs 4.1.0 pyhd8ed1ab_0 conda-forge pooch 1.8.0 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.8.18 hd12c33a_1_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.8 4_cp38 conda-forge pytorch 1.11.0 py3.8_cpu_0 pytorch pytorch-mutex 1.0 cpu pytorch pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge

WeKREthics USA, [1/21/2024 11:29 PM] pyyaml 6.0.1 pypi_0 pypi readline 8.2 h8228510_1 conda-forge regex 2023.12.25 pypi_0 pypi requests 2.31.0 pyhd8ed1ab_0 conda-forge scikit-learn 1.1.2 py38h0b08f9b_0 conda-forge scipy 1.10.1 py38h59b608b_3 conda-forge setuptools 69.0.3 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge tbb 2021.11.0 h00ab1b0_0 conda-forge threadpoolctl 3.2.0 pyha21a80b_0 conda-forge tk 8.6.13 noxft_h4845f30_101 conda-forge tokenizers 0.12.1 pypi_0 pypi tqdm 4.66.1 pypi_0 pypi transformers 4.21.3 pypi_0 pypi typing_extensions 4.9.0 pyha770c72_0 conda-forge urllib3 2.1.0 pyhd8ed1ab_0 conda-forge wheel 0.42.0 pyhd8ed1ab_0 conda-forge xz 5.2.6 h166bdaf_0 conda-forge zstd 1.5.5 hfc55251_0 conda-forge