using the bertmodel in DLPY i cannot find .h5 file for loading weights on the server side

[witkophdw]

I have data science project, where are going to use the bert model in dlpy. we want classify keywords from the description of a movie (imdb).

i am using the jupyter notebook example from dlp. https://github.com/sassoftware/python-dlpy/blob/master/examples/transformers/bert_load_model_quickstart.ipynb

at the end we have to move our local .h5 to the server. we choose the path '/opt/sas/viya/config/data/cas/default/public' , same like the caslib 'public'. so my code for loading the weights are: _# Load and attach the BERT model parameters bert.load_weights('/opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5' , num_target_var=num_tgt_var,

Do not freeze base model weights.

Allow layer updates with model tuning.

freeze_base_model=False )

the error is:

• Warning: /opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5 is the subpath of the caslib, CCDW_CLM_FIXED_ADM. Your action might not be completed successfully as you don't have permission to access the subdirectory of the caslib. To make the directory accessible from CAS, you can recreate the caslib, CCDW_CLM_FIXED_ADM, by calling addCaslib action, and set option subDirectories to True

• ERROR: Invalid string or buffer length • ERROR: Function failed. • ERROR: The action stopped due to errors. • --------------------------------------------------------------------------- • DLPyError Traceback (most recent call last) • Input In [28], in <cell line: 2>() • 1 # Load and attach the BERT model parameters • ----> 2 bert.load_weights('/opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5' , • 3 num_target_var=num_tgt_var, • 4 # Do not freeze base model weights. • 5 # Allow layer updates with model tuning. • 6 freeze_base_model=False • 7 ) •
• File /opt/conda/lib/python3.9/site-packages/dlpy/transformers/bert_model.py:964, in BERT_Model.load_weights(self, path, num_target_var, freeze_base_model, use_gpu, last_frozen_layer) • 961 data_spec = self.get_data_spec(num_target_var) • 963 # attach layer weights • --> 964 super(BERT_Model, self).load_weights(path, • 965 data_spec=data_spec, • 966 use_gpu=use_gpu, • 967 embedding_dim=self._config['hidden_size'])
• 969 # determine which layers to freeze • 970 self._freeze_layers = self._rnn_layer •
• File /opt/conda/lib/python3.9/site-packages/dlpy/network.py:980, in Network.load_weights(self, path, labels, data_spec, label_file_name, label_length, use_gpu, embedding_dim) • 945 ''' • 946 Load the weights from a data file specified by ‘path’ • 947 • (...) • 977 • 978 ''' • 979 if not file_exist_on_server(self.conn, path): • --> 980 raise DLPyError('The file, {}, doesn\'t exist on the server-side.'.format(path)) • 982 server_sep = get_server_path_sep(self.conn) • 984 if server_sep in path: •
• DLPyError: The file, /opt/sas/viya/config/data/cas/default/public/bert-base-uncased.kerasmodel.h5, doesn't exist on the server-side. •

more information about our platform. in the architecture of our Sas Viya platform, they choose to have 2 mn servers(always have copy). /opt/sas/viya/config/data/cas/default/public is pointed to mn01 server and to the other mn server. we are using sas viya 3.5.

[dxq77dxq]

Hello, When you created the caslib CCDW_CLM_FIXED_ADM, please specify subdirectories=True, otherwise you won't have permission to access the data in the subfolders.

[witkophdw]

this caslib has nothing to do with my path. it is caslib connecto to oracle. i think it choose the caslib because it is the first caslib alphabetically. the problem is the path where i store my weights. in our environment we have 2 mn servers. that might be the problem.

[dxq77dxq]

Hello, This issue is the same as https://github.com/sassoftware/python-dlpy/issues/330 and was fixed by https://github.com/sassoftware/python-dlpy/pull/356. We don't have a new release that includes the fix yet. You can git clone the repo and use the developer's version.