Closed WEIYanbin1999 closed 1 year ago
The problem has solved by: However, I don't know why you padding '_delWE' to the pretrain parameter path
Sorry for the late reply. Your way of dealing with it is okay. In fact, we also provide a file "get_pretrained_KGTransformer_parameters.py", running it can obtain the model (suffix is "_delWE") that can be loaded. Hope it will help you.
While I got the pretrain model, they are stored under the BIG/ dictionary. Like this: model_layer-4_hidden-768_heads-12_seq-126_textE-cls_t0-1.0_t1-1.0_t2-1.0.ep9
However, while I run the downstreaming task code, it always mention: logger : INFO cannot load pretrained parameters.
I found there are segment of code in downstraming task python script:
While running code in try: the pretrain_save_path is empty. I try to modify it as model_layer-4_hidden-768_heads-12_seq-126_textE-cls_t0-1.0_t1-1.0_t2-1.0.ep6,7,8,9, but the parameter size mismatch.
So I have a question: where exactly the code utilize and load the parameters of pretrained encoder?