Closed xiaoguzai closed 3 years ago
looks like an error in the docu - try passing any additional keras parameters directly to the BertModelLayer
constructor not the Params
:
from bert import BertModelLayer
l_bert = BertModelLayer(**BertModelLayer.Params(
vocab_size = 16000, # embedding params
use_token_type = True,
use_position_embeddings = True,
token_type_vocab_size = 2,
num_layers = 12, # transformer encoder params
hidden_size = 768,
hidden_dropout = 0.1,
intermediate_size = 4*768,
intermediate_activation = "gelu",
adapter_size = None, # see arXiv:1902.00751 (adapter-BERT)
shared_layer = False, # True for ALBERT (arXiv:1909.11942)
embedding_size = None, # None for BERT, wordpiece embedding size for ALBERT
),
name = "bert" # any other Keras layer params
)
@kpe
Hi, when I run the gpu
moive review example, the python script is here: https://gist.github.com/lijiansong/95c84fbd0bddf82590203c3473f333e9
raise the following errors:
Traceback (most recent call last):
File "gpu_movie_reviews.py", line 218, in <module>
model = create_model(data.max_seq_len, adapter_size=adapter_size)
File "gpu_movie_reviews.py", line 183, in create_model
bert_params = map_stock_config_to_params(bc)
File "/home/extend/work-space/memory/tensorflow-2.1.3/ml-examples/dl/bert-for-tf2/bert/loader.py", line 155, in map_stock_config_to_params
shared_layer=bc.embedding_size is not None,
File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 95, in __init__
self.update(kwargs) # override with kwargs
File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 102, in update
self[key] = arg[key]
File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 120, in __setitem__
"in Params instance '{}'".format(key, self.__class__.__name__))
AttributeError: Setting unexpected parameter 'hidden_size' in Params instance 'Params'
My TensorFlow version is 2.1.3
params_flow: 0.7.4
params: 0.9.0
When i am ` from bert import BertModelLayer
l_bert = BertModelLayer(**BertModelLayer.Params( vocab_size = 16000, # embedding params use_token_type = True, use_position_embeddings = True, token_type_vocab_size = 2,
num_layers = 12, # transformer encoder params hidden_size = 768, hidden_dropout = 0.1, intermediate_size = 4*768, intermediate_activation = "gelu",
adapter_size = None, # see arXiv:1902.00751 (adapter-BERT)
shared_layer = False, # True for ALBERT (arXiv:1909.11942) embedding_size = None, # None for BERT, wordpiece embedding size for ALBERT
name = "bert" # any other Keras layer params )) ` raise error Setting unexpected parameter 'name' in Params instance 'Params'