search

santhoshkolloju / Abstractive-Summarization-With-Transfer-Learning

Abstractive summarisation using Bert as encoder and Transformer Decoder
406 stars 98 forks source link

ValueError during the init of pretrained BERT #29

Open shy1992 opened 5 years ago

shy1992 commented 5 years ago

Hello! I tried your code in a google colab and i encountered a problem i wasn't able to solve. During the inititalization of the Bert encoder in your ipynb: https://github.com/santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning/blob/master/BERT_SUMM.ipynb

in cell 15 there occurs the following error

`Intializing the Bert Encoder Graph loading the bert pretrained weights

ValueError Traceback (most recent call last)

in () 35 init_checkpoint = os.path.join(bert_pretrain_dir, 'bert_model.ckpt') 36 #init_checkpoint = "gs://cloud-tpu-checkpoints/bert/uncased_L-12_H-768_A-12/bert_model.ckpt" ---> 37 model_utils.init_bert_checkpoint(init_checkpoint) 5 frames /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py in _init_from_checkpoint(ckpt_dir_or_file, assignment_map) 344 "Assignment map with scope only name {} should map to scope only " 345 "{}. Should be 'scope/': 'other_scope/'.".format( --> 346 scopes, tensor_name_in_ckpt)) 347 # If scope to scope mapping was provided, find all variables in the scope 348 # and create variable to variable mapping. ValueError: Assignment map with scope only name bert/position_embeddings should map to scope only bert/embeddings/position_embeddings. Should be 'scope/': 'other_scope/'.` i checked the github repo of texar already and found the post: https://github.com/asyml/texar/issues/127 Basically the code for the encoder and decoder changed in the newer version of texar but i dont know how to adjust the code.
santhoshkolloju commented 5 years ago

Clone the older version of code it will be there in releases of texar repo

shy1992 commented 5 years ago

So i tried the following:

`!wget https://github.com/asyml/texar/archive/v0.2.0.zip -P texar_repo; !unzip texar_repo/v0.2.0.zip -d texar_repo if not 'texar_repo' in sys.path: sys.path += ['texar_repo']

!mv texar_repo/texar-0.2.0/* texar_repo`

and the installation of the old texar version: !cd texar_repo && pip install -e .

the error still occurs

santhoshkolloju commented 5 years ago

Please download. 0.1.0

shy1992 commented 5 years ago

Hello thank you for your reply. i got it to work by just copy the repo the thing i got wrong was that texar doesnt need to be installed i guess with the command: sys.path += ['texar_repo'] you get also access to the texar module so pip install is not required.

Thank you for your help! Amazing work btw its great to get insights about a complex tensorflow project.

Off-Topic: Do you have any good ressources to learn tensorflow or did you just read the tensorflow documentation?