search

sassoftware / python-dlpy

The SAS Deep Learning Python (DLPy) package provides the high-level Python APIs to deep learning methods in SAS Visual Data Mining and Machine Learning. It allows users to build deep learning models using friendly Keras-like APIs.
Apache License 2.0
224 stars 131 forks source link

cannot load weights of bert model when i use multiple targets (target sequence) #392

Open witkophdw opened 1 year ago

witkophdw commented 1 year ago

i have project where i have to predict multiple keywords based on description of a movie.

I am able to train the bert model with 1 (binary target) , but I have train the model with multiple (431) binary targets. I got an error when I try to load the weight to cas. parameter:

bert.load_weights('/opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5' , num_target_var=num_tgt_var,

Do not freeze base model weights.

              # Allow layer updates with model tuning.
              freeze_base_model=False
             )

error: ERROR: Target sequence for text input is not supported. ERROR: The action stopped due to errors.

DLPyError Traceback (most recent call last) Input In [17], in <cell line: 1>() ----> 1 bert.load_weights('/opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5' , 2 num_target_var=num_tgt_var, 3 # Do not freeze base model weights. 4 # Allow layer updates with model tuning. 5 freeze_base_model=False 6 )

File ~/python-dlpy/dlpy/transformers/bert_model.py:967, in BERT_Model.load_weights(self, path, num_target_var, freeze_base_model, use_gpu, last_frozen_layer) 964 data_spec = self.get_data_spec(num_target_var) 966 # attach layer weights --> 967 super(BERT_Model, self).load_weights(path, 968 data_spec=data_spec, 969 use_gpu=use_gpu, 970 embedding_dim=self._config['hidden_size'])
972 # determine which layers to freeze 973 self._freeze_layers = self._rnn_layer

File ~/python-dlpy/dlpy/network.py:1003, in Network.load_weights(self, path, labels, data_spec, label_file_name, label_length, use_gpu, embedding_dim) 1000 self.load_weights_from_caffe(path, labels=labels, data_spec=data_spec, label_file_name=label_file_name, 1001 label_length=label_length) 1002 elif file_name.lower().endswith('kerasmodel.h5'): -> 1003 self.load_weights_from_keras(path, labels=labels, data_spec=data_spec, label_file_name=label_file_name, 1004 label_length=label_length, use_gpu=use_gpu, embedding_dim=embedding_dim) 1005 elif file_name.lower().endswith('onnxmodel.h5'): 1006 self.load_weights_from_keras(path, labels=labels, data_spec=data_spec, label_file_name=label_file_name, 1007 label_length=label_length, use_gpu=use_gpu, embedding_dim=embedding_dim)

File ~/python-dlpy/dlpy/network.py:1077, in Network.load_weights_from_keras(self, path, labels, data_spec, label_file_name, label_length, use_gpu, embedding_dim) 1073 self.load_weights_from_file_with_labels(path=path, format_type='KERAS', data_spec=data_spec, 1074 label_file_name=label_file_name, label_length=label_length, 1075 use_gpu=use_gpu, embedding_dim=embedding_dim) 1076 else: -> 1077 self.load_weights_from_file(path=path, format_type='KERAS', data_spec=data_spec, use_gpu=use_gpu, 1078 embedding_dim=embedding_dim)

File ~/python-dlpy/dlpy/network.py:1187, in Network.load_weights_from_file(self, path, format_type, data_spec, use_gpu, embedding_dim) 1185 for msg in rt.messages: 1186 print(msg) -> 1187 raise DLPyError('Cannot import model weights, there seems to be a problem.') 1189 # create attributes if necessary 1190 if not has_data_spec:

DLPyError: Cannot import model weights, there seems to be a problem.

dxq77dxq commented 1 year ago

Hello, Our developers confirmed that target sequence for text input is supported for Viya4.0. If you're using Viya3.5, you'll see this error message.

witkophdw commented 1 year ago

yes, i am working in sas viya 3.5, so i cannot use the general solution of dlpy. we have sas viya 4, but it is in a pilot phase right now. so I have to wait.