NTMC-Community / MatchZoo

Facilitating the design, comparison and sharing of deep text matching models.
Apache License 2.0
3.83k stars 899 forks source link

训练完的模型怎么使用? #725

Closed wang7liang closed 5 years ago

wang7liang commented 5 years ago

您好: 请问如何使用训练好的模型进行预测?

我尝试使用如下代码进行预测 train_df = pd.read_csv('data/train.csv', encoding = 'utf-8') train_dp_raw = mz.data_pack.pack(train_df)

test_df = pd.read_csv('data/test.csv', encoding = 'utf-8') test_dp_raw = mz.data_pack.pack(test_df)

preprocessor = mz.preprocessors.BasicPreprocessor() train_dp_processed = preprocessor.fit_transform(train_dp_raw) test_dp_processed = preprocessor.transform(test_dp_raw)

model = mz.load_model('save/cknrmModel03') pred_x, label = test_dp_processed.unpack() pred_y = model.predict(pred_x, len(label)) 但发现经过BasicPreprocessor处理后的数据,相同的词在词典中的index每次都是不一样的。 这样的话,是否无法准确预测,有什么办法可以解决吗?

ZizhenWang commented 5 years ago

你好,你可以使用preprocessor.save(path)保存fit好的预处理器,再次使用时通过preprocessor = load_preprocessor(path)加载 https://github.com/NTMC-Community/MatchZoo/blob/2547f9d1b302d0f166508ba39fa659dfa210a276/matchzoo/engine/base_preprocessor.py#L131-L141

wang7liang commented 5 years ago

你好,你可以使用preprocessor.save(path)保存fit好的预处理器,再次使用时通过preprocessor = load_preprocessor(path)加载 MatchZoo/matchzoo/engine/base_preprocessor.py

Lines 131 to 141 in 2547f9d

def load_preprocessor(dirpath: typing.Union[str, Path]) -> 'mz.DataPack': """ Load the fitted context. The reverse function of :meth:save.

 :param dirpath: directory path of the saved model. 
 :return: a :class:`DSSMPreprocessor` instance. 
 """ 
 dirpath = Path(dirpath) 

 data_file_path = dirpath.joinpath(BasePreprocessor.DATA_FILENAME) 
 return dill.load(open(data_file_path, 'rb'))

非常感谢,解决了我遇到的问题

bgkbz commented 4 years ago

非常感谢!预测效果不理想,也没发现这个问题,终于解决了!