I changed these parameters (hidden_size,filter_size,num_heads,num_encoder_layers,num_decoder_layers) in public_params and pegasus params to the right values (768,3072,12,12,12).
I hope you would help me to train the pegasus base. what are other parameters that should I change?
Hi I try to train pegasus base (num of heads = 12 and hidden state = 768) using TPU but I got the following error:
[[node TPUReplicateMetadata (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]] (7) Invalid argument: Run-time shape mismatch for TPUExecute argument[1] (cluster/_variable_copy/_930_G11162). Expected element_type: F32 dimensions: 768 layout { minor_to_major: 0 format: DENSE } is_dynamic_dimension: false ; got element_type: F32 dimensions: 1024 layout { minor_to_major: 0 format: DENSE } is_dynamic_dimension: false
I changed these parameters (hidden_size,filter_size,num_heads,num_encoder_layers,num_decoder_layers) in public_params and pegasus params to the right values (768,3072,12,12,12). I hope you would help me to train the pegasus base. what are other parameters that should I change?