search

yongzhuo / Keras-TextClassification

中文长文本分类、短句子分类、多标签分类、两句子相似度（Chinese Text Classification of Keras NLP, multi-label classify, or sentence classify, long or short），字词句向量嵌入层（embeddings）和网络层（graph）构建基类，FastText，TextCNN，CharCNN，TextRNN, RCNN, DCNN, DPCNN, VDCNN, CRNN, Bert, Xlnet, Albert, Attention, DeepMoji, HAN, 胶囊网络-CapsuleNet, Transformer-encode, Seq2seq, SWEM, LEAM, TextGCN
https://blog.csdn.net/rensihui
MIT License
1.77k stars 406 forks source link

what's the problem? #38

Closed shockstonem closed 4 years ago

shockstonem commented 4 years ago

pip install Keras-TextClassification -i https://pypi.tuna.tsinghua.edu.cn/simple

Using cached https://pypi.tuna.tsinghua.edu.cn/packages/78/17/f611fc9bc62d072e069b1bc504d299dc0bd8b1bb8043f7b2252f407eec9e/Keras_TextClassification-0.1.6-py2.py3-none-any.whl (218 kB) ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.12.0 (from Keras-TextClassification) (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0rc0, 1.15.0rc1, 1.15.0rc2, 1.15.0rc3, 1.15.0, 1.15.2, 1.15.3, 2.0.0a0, 2.0.0b0, 2.0.0b1, 2.0.0rc0, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc0, 2.1.0rc1, 2.1.0rc2, 2.1.0, 2.1.1, 2.2.0rc0, 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0) ERROR: No matching distribution found for tensorflow-gpu==1.12.0 (from Keras-TextClassification)

pip install Keras-TextClassification

Collecting Keras-TextClassification Downloading Keras_TextClassification-0.1.6-py2.py3-none-any.whl (218 kB) |████████████████████████████████| 218 kB 25 kB/s Requirement already satisfied: pandas>=0.23.4 in d:\anaconda3\lib\site-packages (from Keras-TextClassification) (0.24.2) Collecting keras-xlnet>=0.16.0 Downloading keras-xlnet-0.18.0.tar.gz (21 kB) Collecting keras-bert>=0.80.0 Downloading keras-bert-0.86.0.tar.gz (26 kB) Requirement already satisfied: gensim>=3.7.1 in d:\anaconda3\lib\site-packages (from Keras-TextClassification) (3.8.3) Requirement already satisfied: jieba>=0.39 in d:\anaconda3\lib\site-packages (from Keras-TextClassification) (0.39) Collecting passlib>=1.7.1 Downloading passlib-1.7.2-py2.py3-none-any.whl (507 kB) |████████████████████████████████| 507 kB 5.8 kB/s Collecting keras-adaptive-softmax>=0.6.0 Downloading keras-adaptive-softmax-0.6.0.tar.gz (16 kB) Requirement already satisfied: scikit-learn>=0.19.1 in d:\anaconda3\lib\site-packages (from Keras-TextClassification) (0.20.3) Collecting keras==2.2.4 Downloading Keras-2.2.4-py2.py3-none-any.whl (312 kB) |████████████████████████████████| 312 kB 32 kB/s ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.12.0 (from Keras-TextClassification) (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0rc0, 1.15.0rc1, 1.15.0rc2, 1.15.0rc3, 1.15.0, 1.15.2, 1.15.3, 2.0.0a0, 2.0.0b0, 2.0.0b1, 2.0.0rc0, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc0, 2.1.0rc1, 2.1.0rc2, 2.1.0, 2.1.1, 2.2.0rc0, 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0) ERROR: No matching distribution found for tensorflow-gpu==1.12.0 (from Keras-TextClassification)

yongzhuo commented 4 years ago

你直接pip install Keras-TextClassification呢，不要用tsinghua镜像源，看着像是没有tensorflow-gpu==1.12.0的版本

shockstonem commented 4 years ago

两种我都试了，直接pip install Keras-TextClassification就是下面这个ERROR 我电脑装的的tensorflow-gpu是1.13.1版本，怎么提示found for tensorflow-gpu==1.12.0 ? 然后你这个需要的是哪个版本呢?

yongzhuo commented 4 years ago

是需要这个版本的，keras=2.2.4，否则有的可能不支持。 ------------------ 原始邮件 ------------------ 发件人: "shockstonem"<notifications@github.com> 发送时间: 2020年8月25日(星期二) 下午5:42 收件人: "yongzhuo/Keras-TextClassification"<Keras-TextClassification@noreply.github.com>; 抄送: "yongzhuo"<2714618994@qq.com>;"Comment"<comment@noreply.github.com>; 主题: Re: [yongzhuo/Keras-TextClassification] what's the problem? (#38)

shockstonem commented 4 years ago

我的配置：keras=2.2.4
tensorflow-gpu=1.13.1 gensim = 3.83 python = 3.7.3 现在就是pip install Keras-TextClassification，出现上面的问题?不知道什么原因，请指教

yongzhuo commented 4 years ago

keras和tensorflow的版本对不上吧------------------ 原始邮件 ------------------ 发件人: "shockstonem"<notifications@github.com> 发送时间: 2020年8月25日(星期二) 晚上8:20 收件人: "yongzhuo/Keras-TextClassification"<Keras-TextClassification@noreply.github.com>; 抄送: "yongzhuo"<2714618994@qq.com>;"Comment"<comment@noreply.github.com>; 主题: Re: [yongzhuo/Keras-TextClassification] what's the problem? (#38)

shockstonem commented 4 years ago

感谢解决了前面的问题，是keras和tensorflow的版本未对应造成，但是又出现新的问题 按照说明进入目录后python train.py报错：

Traceback (most recent call last): File "train.py", line 21, in from keras_textclassification.conf.path_config import path_baidu_qa_2019_train, path_baidu_qa_2019_valid, \ ImportError: cannot import name 'path_ccks_2020_el_dev' from 'keras_textclassification.conf.path_config' (C:\Users\Charming\Desktop\capsNet\Keras-TextClassification-master\keras_textclassification\conf\path_config.py)

看了源码C:\Users\Charming\Desktop\capsNet\Keras-TextClassification-master\keras_textclassification\conf\path_config.py里面没有定义path_ccks_2020_el_dev，请问如何定义，以及相关数据集有提供吗?

yongzhuo commented 4 years ago

path_ccks_2020_el_dev是冗余的，你去掉就好。用path_baidu_qa_2019_train, path_baidu_qa_2019_valid就好

shockstonem commented 4 years ago

Traceback (most recent call last): File "train.py", line 126, in train(rate=1) File "train.py", line 112, in train ra_ed, rate=rate, shuffle=True) File "C:\Users\Charming\Desktop\capsNet\Keras-TextClassification-master\keras_textclassification\data_preprocess\text_preprocess.py", line 491, in preprocess_label_ques_to_idx ques_1 = data['sentence1'].tolist() File "D:\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2902, in getitem indexer = self.columns.get_loc(key) File "D:\Anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 2891, in get_loc raise KeyError(key) from err KeyError: 'sentence1'

好像指示text_preprocess.py里面的sentence1报错，这个文件里面的sentence1，sentence2代表什么呢

yongzhuo commented 4 years ago

sentence1，sentence2做文本相似度的。

  1. 将trian.py中的

    数据预处理

    pt = PreprocessSim(path_model_dir) 改成 pt = PreprocessText(path_model_dir) 就可以啦

  2. 或者是: m01_FastText, m10_DeepMoji 和 m13_CapsuleNet以前做实验可能有冗余。 其他应该没有问题的。你将其他的train.py复制一下， 改一下引入应该就可以了

    模型图

    from keras_textclassification.m01_FastText.graph import FastTextGraph as Graph