sfwydyc
commented
2 years ago 抱歉，RocketQA工具目前不支持并行
Open zhaoyiyong opened 2 years ago
sfwydyc
commented
2 years ago 抱歉，RocketQA工具目前不支持并行
RussellLuo
commented
1 year ago @zhaoyiyong 作为一种备选方案，可以将 RocketQA 模型转换成 Paddle 推理模型（参考 #73），然后就能实现 多线程并发推理。
在examples/faiss_example中的rocketqa_service.py是个单进程处理的demo，无法适应同时处理多个请求。我将其改为多进程多线程的处理方式，响应就会出现错误，查看错误信息，发现就是调用： q_embs = self._dual_encoder.encode_query(query=[query]) File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/rocketqa/predict/dual_encoder.py", line 145, in encode_query fetch_list=fetch_list) File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1299, in run six.reraise(*sys.exc_info()) File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/six.py", line 719, in reraise raise value File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1295, in run return_merged=return_merged) File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1464, in _run_impl return new_exe.run(list(feed.keys()), fetch_list, return_numpy) File "/home/eyundl/anaconda3/envs/paddle_env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 547, in run tensors = self._new_exe.run(feed_names, fetch_list)._move_to_list() 请问：调用模型计算的时候，不能并行吗？