hankcs / HanLP

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理
https://hanlp.hankcs.com/
Apache License 2.0
33.84k stars 10.12k forks source link

无法加载英语SDP模型(SEMEVAL15_PAS_BIAFFINE_EN) #1902

Closed hyl2001 closed 2 months ago

hyl2001 commented 2 months ago

Describe the bug 无法加载英语SDP模型。

Code to reproduce the issue

import hanlp

hanlp.load('SEMEVAL15_PAS_BIAFFINE_EN')

Describe the current behavior

hanlp无法正常加载英语SDP模型。我卸载了hanlp以及所有依赖,利用pip install hanlp[full]重新安装了hanlp,但是问题依旧。

Expected behavior

希望可以正常加载英语SDP模型。

System information

Other info / logs

Failed to load https://file.hankcs.com/hanlp/sdp/semeval15_biaffine_pas_20200103_152405.zip

================================ERROR LOG BEGINS================================
Traceback (most recent call last):
  File "c:\Users\win11\Desktop\thesis\study\SDP.py", line 3, in <module>
    hanlp.load('SEMEVAL15_PAS_BIAFFINE_EN')
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\__init__.py", line 43, in load
    return load_from_meta_file(save_dir, 'meta.json', verbose=verbose, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\utils\component_util.py", line 186, in load_from_meta_file
    raise e from None
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\utils\component_util.py", line 106, in load_from_meta_file
    obj.load(save_dir, verbose=verbose, **kwargs)
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\common\keras_component.py", line 215, in load
    self.build(**merge_dict(self.config, training=False, logger=logger, **kwargs, overwrite=True, inplace=True))
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\common\keras_component.py", line 225, in build
    self.model = self.build_model(**merge_dict(self.config, training=kwargs.get('training', None),
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\components\parsers\biaffine_parser_tf.py", line 42, in build_model
    pretrained: tf.keras.layers.Embedding = build_embedding(pretrained_embed, self.transform.form_vocab,
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\hanlp\layers\embeddings\util_tf.py", line 44, in build_embedding
    layer: tf.keras.layers.Embedding = tf.keras.utils.deserialize_keras_object(embeddings)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\win11\AppData\Local\Programs\Python\Python311\Lib\site-packages\keras\src\saving\serialization_lib.py", line 720, in deserialize_keras_object
    raise TypeError(
TypeError: <class 'hanlp.layers.embeddings.word2vec_tf.Word2VecEmbeddingTF'> could not be deserialized properly. Please ensure that components that are Python object instances (layers, models, etc.) returned by `get_config()` are explicitly deserialized in the model's `from_config()` method.

config={'class_name': 'HanLP>Word2VecEmbedding', 'config': {'trainable': False, 'embeddings_initializer': 'zero', 'filepath': 'http://downloads.cs.stanford.edu/nlp/data/glove.6B.zip#glove.6B.100d.txt', 'expand_vocab': True, 'lowercase': True, 'normalize': True, 'vocab': <hanlp.common.vocab_tf.VocabTF object at 0x00000172A9695290>}}.

Exception encountered: Error when deserializing class 'Word2VecEmbeddingTF' using config={'trainable': False, 'embeddings_initializer': 'zero', 'filepath': 'http://downloads.cs.stanford.edu/nlp/data/glove.6B.zip#glove.6B.100d.txt', 'expand_vocab': True, 'lowercase': True, 'normalize': True, 'vocab': <hanlp.common.vocab_tf.VocabTF object at 0x00000172A9695290>}.

Exception encountered: 'bool' object has no attribute 'shape'
=================================ERROR LOG ENDS=================================
hankcs commented 2 months ago

Python 3.11不在支持范围内,请使用低版本python:

https://github.com/hankcs/HanLP/blob/master/setup.py#L57

hyl2001 commented 2 months ago

Python 3.11不在支持范围内,请使用低版本python:

https://github.com/hankcs/HanLP/blob/master/setup.py#L57

感谢回复,但是我用了python 3.10,问题依旧,报错和前边说的一样

hankcs commented 2 months ago

tensorflow的兼容性非常差,tensorflow版本也得低一些:

https://github.com/hankcs/HanLP/blob/master/setup.py#L18

hyl2001 commented 2 months ago

感谢解答,问题已经解决了。