search

PaddlePaddle / RocketQA

🚀 RocketQA, dense retrieval for information retrieval and question answering, including both Chinese and English state-of-the-art models.
Apache License 2.0
766 stars 128 forks source link

rocketqs 运行demo代码时候报错 #86

Open zck573693104 opened 1 year ago

zck573693104 commented 1 year ago

paddle-bfloat 0.1.7 paddle-pipelines 0.5.0 paddle2onnx 0.9.6 paddlefsl 1.1.0 paddlenlp 2.5.1 paddleocr 2.5 paddlepaddle-gpu 2.3.2.post116 rocketqa 1.1.0 当运行demo代码时候报错

使用RocketQA预置的模型做预测

import rocketqa

query_list = ["交叉验证的作用"] para_list = ["交叉验证(Cross-validation)主要用于建模应用中，例如PCR 、PLS回归建模中。在给定的建模样本中，拿出大部分样本进行建模型，留小部分样本用刚建立的模型进行预报，并求这小部分样本的预报误差，记录它们的平方加和。"] title_list = ["交叉验证的介绍"]

dual_encoder = rocketqa.load_model(model="zh_dureader_de_v2") # for cpu

q_embs = dual_encoder.encode_query(query=query_list) p_embs = dual_encoder.encode_para(para=para_list, title=title_list)

dot_products = dual_encoder.matching(query=query_list, title=title_list, para=para_list) print (dot_products)

RocketQA model [zh_dureader_de_v2] WARNING:root:paddle.fluid.layers.py_reader() may be deprecated in the near future. Please use paddle.fluid.io.DataLoader.from_generator() instead. [WARNING] 2023-03-11 14:53:52,916 [ io.py: 721]: paddle.fluid.layers.py_reader() may be deprecated in the near future. Please use paddle.fluid.io.DataLoader.from_generator() instead. ---------------------------------------------------------------------------RuntimeError Traceback (most recent call last)/tmp/ipykernel_131/2762825280.py in 8 9 # load model - RocketQA中文检索模型(de)，在DuReader数据集中训练得到 ---> 10 dual_encoder = rocketqa.load_model(model="zh_dureader_de_v2") # for cpu 11 # dual_encoder = rocketqa.load_model(model="v1_marco_de", use_cuda=True, device_id=0, batch_size=16) # for gpu 12 /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/rocketqa/rocketqa.py in load_model(model, use_cuda, device_id, batch_size) 118 119 if model_type[0] == "d": --> 120 encoder = DualEncoder(encoder_conf) 121 elif model_type[0] == "c": 122 encoder = CrossEncoder(encoder_conf) /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/rocketqa/encoder/dual_encoder.py in init(self, conf_path, use_cuda, device_id, batch_size, **kwargs) 90 ernie_config=self.ernie_config, 91 is_prediction=True, ---> 92 share_parameter=args.share_parameter) 93 94 self.test_prog = self.test_prog.clone(for_test=True) /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/rocketqa/model/dual_encoder_predict.py in create_predict_model(args, pyreader_name, ernie_config, is_prediction, task_name, share_parameter) 51 lod_levels=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 52 name=pyreader_name, ---> 53 use_double_buffer=True) 54 55 (src_ids_q, sent_ids_q, pos_ids_q, task_ids_q, input_mask_q, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/io.py in py_reader(capacity, shapes, dtypes, lod_levels, name, use_double_buffer) 727 lod_levels=lod_levels, 728 name=name, --> 729 use_double_buffer=use_double_buffer) 730 731 /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/io.py in _py_reader(capacity, shapes, dtypes, lod_levels, name, use_double_buffer, feed_list) 438 439 var = global_scope().var(queue_name) --> 440 feed_queue = core.init_lod_tensor_blocking_queue(var, capacity, False) 441 442 startup_blk = default_startup_program().currentblock() RuntimeError: (AlreadyExists) LoDTensorBlockingQueueHolder::InitOnce() can only be called once [Hint: Expected queue == nullptr, but received queue_ != nullptr.] (at /paddle/paddle/fluid/operators/reader/lod_tensor_blocking_queue.h:207)

ylf-Ng commented 1 year ago

RocketQA采用静态图实现，因此在JupyterNotebook里面不能重复运行dual_encoder = rocketqa.load_model(model="zh_dureader_de_v2")