关于您实现的Albert和Xlnet的两个问题

[LinXin04]

您好！首先感谢您的贡献。但是在实现的时候发现会有两个地方报错，一直不明白。 实现Albert时提示：KeyError: 'vocab_size'. 实现xlnet时提示：AttributeError: Layer Attention-Normal-24 has multiple inbound nodes, hence the notion of "layer output" is ill-defined. Use get_output_at(node_index) instead.

[yongzhuo]

1. 只支持这两个预训练模型，其他的不支持， albert_base_zh：brightmart训练的albert, 地址为：https://github.com/brightmart/albert_zh

chinese_xlnet_mid_L-24_H-768_A-12：哈工大预训练的中文xlnet模型 地址为： https://github.com/ymcui/Chinese-PreTrained-XLNet

2. 如果你想要支持其他的预训练模型，比如12层的xlnet，请前往embedding修改输出层， 或者是使用默认输出，超参数配置，示例如下:

   'embedding': {'layer_indexes': [], # 这里为空表示默认输出
                      'xlnet_embed':{'attention_type': 'bi',  # or 'uni'
                                    'memory_len': 0,
                                    'target_len': 50, #这里最好和len_max保持一致
                                  },
                       },

[LinXin04]

您好，我用的就是您说的这两个模型，在windows下运行的是报错的。

[yongzhuo]

请确认keras和tf版本，我重新下载项目代码，win10下跑是没有问题的

[LinXin04]

你好，请问您用的是什么版本呢？ 我用的是 Keras： 2.2.4 TensorFlow：1.14.0

[yongzhuo]

请看：requirements.txt keras==2.2.4 tensorflow==1.12.0

最新的项目代码，你跑一下m00_Xlnet/train.py看看可不可以呢？

[LinXin04]

我换了您的版本，运行m00_Xlnet/train.py，还是报如下错误。 Traceback (most recent call last): File "E:/Code/Keras-TextClassification-master/keras_textclassification/m00_Xlnet/train.py", line 100, in train(rate=1) # sample条件下设为1,否则训练语料可能会很少 File "E:/Code/Keras-TextClassification-master/keras_textclassification/m00_Xlnet/train.py", line 81, in train graph = Graph(hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\m00_Xlnet\graph.py", line 29, in init super().init(hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\graph.py", line 62, in init self.create_model(hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\m00_Xlnet\graph.py", line 37, in create_model super().create_model(hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\graph.py", line 88, in create_model self.word_embedding = Embeddings(hyper_parameters=hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 275, in init super().init(hyper_parameters) File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 68, in init self.build() File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 330, in build for lay in self.layer_indexes] File "E:\Code\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 330, in for lay in self.layer_indexes] File "C:\Users\linxin1\AppData\Local\Programs\Python\Python37\lib\site-packages\keras\engine\base_layer.py", line 813, in output ' has multiple inbound nodes, ' AttributeError: Layer Attention-Normal-24 has multiple inbound nodes, hence the notion of "layer output" is ill-defined. Use get_output_at(node_index) instead.

[LinXin04]

实现Albert时提示如下错误 Traceback (most recent call last): File "E:/Keras-TextClassification-master/keras_textclassification/m00_Albert/train.py", line 100, in train(rate=1) File "E:/Keras-TextClassification-master/keras_textclassification/m00_Albert/train.py", line 81, in train graph = Graph(hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\m00_Albert\graph.py", line 28, in init super().init(hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\base\graph.py", line 62, in init self.create_model(hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\m00_Albert\graph.py", line 36, in create_model super().create_model(hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\base\graph.py", line 88, in create_model self.word_embedding = Embeddings(hyper_parameters=hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 362, in init super().init(hyper_parameters) File "E:\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 68, in init self.build() load bert model start! File "E:\Keras-TextClassification-master\keras_textclassification\base\embedding.py", line 376, in build output_layers = self.layer_indexes) File "E:\Keras-TextClassification-master\keras_textclassification\keras_layers\albert\albert.py", line 245, in load_brightmart_albert_zh_checkpoint _set_if_not_existed('token_num', config['vocab_size']) KeyError: 'vocab_size'

[LinXin04]

另外，bert可以成功运行

[LinXin04]

是用pip install安装的。。。

[yongzhuo]

pip安装的版本有点老，有的有问题没有更新 此外，今天修复了albert层数bug和xlnet的输入bug, xlnet选择层数问题我还得看看

[LinXin04]

好的，期待大佬的更新！

[yongzhuo]

已修复, 你试试最新的代码呢

[aszhanghuali]

@LinXin04 请问你的问题解决了吗？我运行m00_Albert/train.py也是同样地问题：KeyError: 'vocab_size'。并且我的keras=2.2.4，tensorflow==1.12.0。

[yongzhuo]

@aszhanghuali

1. albert_base_zh/目录下 init.py albert_config.json albert_model.ckpt.data-00000-of-00001 albert_model.ckpt.index albert_model.ckpt.meta 全部改为 bert_config.json bert_model.ckpt.data-00000-of-00001 bert_model.ckpt.index bert_model.ckpt.meta

2.checkpoint文件中 model_checkpoint_path: "albert_model.ckpt" all_model_checkpoint_paths: "albert_model.ckpt" 改为 model_checkpoint_path: "bert_model.ckpt" all_model_checkpoint_paths: "bert_model.ckpt"

[raining-dev]

AttributeError: 'list' object has no attribute 'transpose'