kpe / bert-for-tf2

A Keras TensorFlow 2.0 implementation of BERT, ALBERT and adapter-BERT.
https://github.com/kpe/bert-for-tf2
MIT License
803 stars 193 forks source link

Setting unexpected parameter 'name' in Params instance 'Params' #87

Closed xiaoguzai closed 3 years ago

xiaoguzai commented 3 years ago

When i am ` from bert import BertModelLayer

l_bert = BertModelLayer(**BertModelLayer.Params( vocab_size = 16000, # embedding params use_token_type = True, use_position_embeddings = True, token_type_vocab_size = 2,

num_layers = 12, # transformer encoder params hidden_size = 768, hidden_dropout = 0.1, intermediate_size = 4*768, intermediate_activation = "gelu",

adapter_size = None, # see arXiv:1902.00751 (adapter-BERT)

shared_layer = False, # True for ALBERT (arXiv:1909.11942) embedding_size = None, # None for BERT, wordpiece embedding size for ALBERT

name = "bert" # any other Keras layer params )) ` raise error Setting unexpected parameter 'name' in Params instance 'Params'

kpe commented 3 years ago

looks like an error in the docu - try passing any additional keras parameters directly to the BertModelLayer constructor not the Params:

from bert import BertModelLayer

l_bert = BertModelLayer(**BertModelLayer.Params(
    vocab_size = 16000, # embedding params
    use_token_type = True,
    use_position_embeddings = True,
    token_type_vocab_size = 2,

    num_layers = 12, # transformer encoder params
    hidden_size = 768,
    hidden_dropout = 0.1,
    intermediate_size = 4*768,
    intermediate_activation = "gelu",

    adapter_size = None, # see arXiv:1902.00751 (adapter-BERT)

    shared_layer = False, # True for ALBERT (arXiv:1909.11942)
    embedding_size = None, # None for BERT, wordpiece embedding size for ALBERT
  ),
  name = "bert" # any other Keras layer params
)
lijiansong commented 3 years ago

@kpe Hi, when I run the gpu moive review example, the python script is here: https://gist.github.com/lijiansong/95c84fbd0bddf82590203c3473f333e9 raise the following errors:

Traceback (most recent call last):
  File "gpu_movie_reviews.py", line 218, in <module>
    model = create_model(data.max_seq_len, adapter_size=adapter_size)
  File "gpu_movie_reviews.py", line 183, in create_model
    bert_params = map_stock_config_to_params(bc)
  File "/home/extend/work-space/memory/tensorflow-2.1.3/ml-examples/dl/bert-for-tf2/bert/loader.py", line 155, in map_stock_config_to_params
    shared_layer=bc.embedding_size is not None,
  File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 95, in __init__
    self.update(kwargs)            # override with kwargs
  File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 102, in update
    self[key] = arg[key]
  File "/home/extend/work-space/anaconda2/envs/tf2-source/lib/python3.5/site-packages/params/params.py", line 120, in __setitem__
    "in Params instance '{}'".format(key, self.__class__.__name__))
AttributeError: Setting unexpected parameter 'hidden_size' in Params instance 'Params'

My TensorFlow version is 2.1.3 params_flow: 0.7.4 params: 0.9.0