search

baidu / AnyQ

FAQ-based Question Answering System
Apache License 2.0
2.58k stars 672 forks source link

compile error #16

Closed daiw10 closed 6 years ago

daiw10 commented 6 years ago

[100%] Built target paddle_fluid [100%] Built target paddle_fluid_shared copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/inference/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/inference copying /home/daiwei/AnyQ/build/third_party/install/paddle/paddle/fluid/inference/libpaddle_fluid. -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/inference [100%] Built target inference_lib [100%] Built target profiler_py_proto_init Copy generated python proto into directory paddle/fluid/proto/profiler. [100%] Built target profiler_py_proto copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/platform/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/platform copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/platform/dynload/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/platform/dynload copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/platform/details/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/platform/details [100%] Built target platform_lib copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/string/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/string copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/string/tinyformat/*.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/string/tinyformat [100%] Built target string_lib [100%] Built target inference_lib_dist [ 15%] Completed 'extern_paddle' [ 15%] Built target extern_paddle make: *** [all] Error 2

My gcc version is 4.8.5 and cmake 3.2.0 Many thanks

zhanghan1992 commented 6 years ago

你贴的错误信息不完整，贴一下完整的错误信息

daiw10 commented 6 years ago

完整的太长了orz。。截了开头和结尾 这样可以么，中间都是一堆boost的信息， 谢谢

make -j8 [ 1%] Creating directories for 'extern_jsoncpp' [ 2%] Creating directories for 'extern_leveldb' [ 2%] Creating directories for 'extern_eigen' [ 2%] Creating directories for 'extern_paddle' [ 3%] Creating directories for 'extern_gtest' [ 3%] Creating directories for 'extern_curl' [ 3%] Creating directories for 'extern_boost' [ 3%] Creating directories for 'extern_xgboost' [ 3%] Performing download step (git clone) for 'extern_jsoncpp' -- Avoiding repeated git clone, stamp file is up to date: '/home/daiwei/AnyQ/build/third_party/jsoncpp/src/extern_jsoncpp-stamp/extern_jsoncpp-gitclone-lastrun.txt' [ 5%] Performing download step for 'extern_leveldb' [ 5%] Performing download step for 'extern_boost' [ 5%] Performing download step (git clone) for 'extern_paddle' [ 6%] Performing download step for 'extern_xgboost' [ 6%] Performing download step for 'extern_eigen' [ 7%] Performing download step for 'extern_curl' fatal: destination path 'xgboost' already exists and is not an empty directory. fatal: destination path 'eigen' already exists and is not an empty directory. make[2]: [third_party/xgboost/src/extern_xgboost-stamp/extern_xgboost-download] Error 128 make[2]: make[1]: [CMakeFiles/extern_xgboost.dir/all] Error 2 [third_party/eigen/src/extern_eigen-stamp/extern_eigen-download] Error 128 --2018-07-31 15:48:12-- https://jaist.dl.sourceforge.net/project/boost/boost/1.41.0/boost_1_41_0.tar.gz make[1]: Waiting for unfinished jobs.... make[1]: [CMakeFiles/extern_eigen.dir/all] Error 2 fatal: destination path 'leveldb' already exists and is not an empty directory. make[2]: [third_party/leveldb/src/extern_leveldb-stamp/extern_leveldb-download] Error 128 --2018-07-31 15:48:12-- https://curl.haxx.se/download/curl-7.60.0.tar.gz make[1]: *** [CMakeFiles/extern_leveldb.dir/all] Error 2 -- Avoiding repeated git clone, stamp file is up to date: '/home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle-stamp/extern_paddle-gitclone-lastrun.txt' Resolving jaist.dl.sourceforge.net (jaist.dl.sourceforge.net)... [ 7%] No update step for 'extern_paddle' Resolving curl.haxx.se (curl.haxx.se)... [ 8%] No patch step for 'extern_paddle' [ 8%] Performing configure step for 'extern_paddle' 150.65.7.130, 2001:df0:2ed:feed::feed Connecting to jaist.dl.sourceforge.net (jaist.dl.sourceforge.net)|150.65.7.130|:443... -- Found Paddle host system: centos, version: -- Found Paddle host system's CPU: 8 cores -- CXX compiler: /usr/bin/c++, version: GNU 4.8.5 -- C compiler: /usr/bin/cc, version: GNU 4.8.5 -- MKLML_VER: mklml_lnx_2018.0.3.20180406, MKLML_URL: http://paddlepaddledeps.cdn.bcebos.com/mklml_lnx_2018.0.3.20180406.tgz connected. [ 8%] No patch step for 'extern_jsoncpp' [ 10%] Performing download step (git clone) for 'extern_gtest' [ 10%] No update step for 'extern_jsoncpp' -- Avoiding repeated git clone, stamp file is up to date: '/home/daiwei/AnyQ/build/third_party/gtest/src/extern_gtest-stamp/extern_gtest-gitclone-lastrun.txt' [ 11%] Performing configure step for 'extern_jsoncpp' [ 11%] No patch step for 'extern_gtest' [ 12%] No update step for 'extern_gtest' [ 13%] Performing configure step for 'extern_gtest' loading initial cache file /home/daiwei/AnyQ/build/third_party/gtest/tmp/extern_gtest-cache-.cmake -- JsonCpp Version: 1.8.4 -- Configuring done -- Configuring done HTTP request sent, awaiting response... 302 Found Location: http://downloads.sourceforge.net/mirrorproblem?failedmirror=jaist.dl.sourceforge.net [following] --2018-07-31 15:48:12-- http://downloads.sourceforge.net/mirrorproblem?failedmirror=jaist.dl.sourceforge.net Resolving downloads.sourceforge.net (downloads.sourceforge.net)... -- Generating done -- Generating done 216.105.38.13 Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.105.38.13|:80... -- Build files have been written to: /home/daiwei/AnyQ/build/third_party/gtest/src/extern_gtest-build [ 13%] Performing build step for 'extern_gtest' -- Build files have been written to: /home/daiwei/AnyQ/build/third_party/jsoncpp/src/extern_jsoncpp [ 14%] Performing build step for 'extern_jsoncpp' make[3]: warning: jobserver unavailable: using -j1. Add +' to parent make rule. [ 18%] Built target gtest [ 54%] Built target gmock [ 81%] Built target gmock_main [100%] Built target gtest_main [ 44%] Built target jsoncpp_lib_static [ 15%] Performing install step for 'extern_gtest' [ 66%] Built target jsontestrunner_exe [ 18%] Built target gtest [ 54%] Built target gmock_main [100%] Built target jsoncpp_test [ 81%] Built target gmock [ 15%] Performing install step for 'extern_jsoncpp' [100%] Built target gtest_main make[3]: warning: jobserver unavailable: using -j1. Add+' to parent make rule. Install the project...

中间省略

[ 29%] Built target adamax_op [ 29%] Built target concurrency [ 29%] Built target channel_recv_op [ 29%] Built target adadelta_op [ 29%] Built target activation_op [ 30%] Built target channel_send_op [ 30%] Built target average_accumulates_op [ 31%] Built target load_combine_op [ 31%] Built target save_op [ 31%] Built target clip_op [ 31%] Built target dropout_op [ 31%] Built target edit_distance_op [ 31%] Built target save_combine_op [ 31%] Built target arg_min_op [ 32%] Built target decayed_adagrad_op [ 32%] Built target detection_map_op [ 32%] Built target fill_constant_batch_size_like_op [ 32%] Built target huber_loss_op [ 32%] Built target vol2col [ 32%] Built target im2col [ 32%] Built target conv_op [ 32%] Built target elementwise_max_op [ 32%] Built target sequence_padding [ 32%] Built target sequence_scale [ 32%] Built target warpctc_op [ 32%] Built target batch_norm_op [ 34%] Built target parallel_do_op [ 34%] Built target while_op [ 34%] Built target sequence_slice_op [ 34%] Built target context_project [ 34%] Built target sequence_conv_op [ 34%] Built target adam_op [ 34%] Built target softmax [ 34%] Built target softmax_op [ 34%] Built target lookup_sparse_table_op [ 34%] Built target cumsum_op [ 34%] Built target accuracy_op [ 34%] Built target transpose_op [ 34%] Built target cos_sim_functor [ 34%] Built target cos_sim_op [ 34%] Built target feed_op [ 34%] Built target reduce_mean_op [ 34%] Built target activation_functions [ 34%] Built target gru_compute [ 35%] Built target sequence2batch [ 35%] Built target gru_op [ 35%] Built target conv_transpose_op [ 35%] Built target ctc_align_op [ 36%] Built target reduce_max_op [ 36%] Built target sequence_softmax_op [ 36%] Built target max_sequence_len_op [ 36%] Built target maxouting [ 36%] Built target maxout_op [ 36%] Built target cross_entropy [ 36%] Built target cross_entropy_op [ 36%] Built target cast_op [ 36%] Built target l1_norm_op [ 36%] Built target mul_op [ 36%] Built target auc_op [ 36%] Built target gaussian_random_batch_size_like_op [ 36%] Built target softmax_with_cross_entropy_op [ 36%] Built target sequence_pooling [ 37%] Built target sequence_pool_op [ 38%] Built target momentum_op [ 38%] Built target expand_op [ 38%] Built target is_empty_op [ 38%] Built target label_smooth_op [ 38%] Built target lod_rank_table_op [ 38%] Built target array_to_lod_tensor_op [ 39%] Built target assign_value_op [ 39%] Built target lookup_table_op [ 39%] Built target lod_tensor_to_array_op [ 39%] Built target recurrent_op [ 40%] Built target one_hot_op [ 40%] Built target arg_max_op [ 40%] Built target print_op [ 40%] Built target load_op [ 40%] Built target concat [ 40%] Built target concat_op [ 40%] Built target lstm_compute [ 40%] Built target lstm_op [ 40%] Built target assign_op [ 40%] Built target sequence_expand_op [ 40%] Built target channel_create_op [ 40%] Built target squared_l2_distance_op [ 40%] Built target elementwise_sub_op [ 40%] Built target sequence_reshape_op [ 40%] Built target pooling [ 40%] Built target pool_op [ 40%] Built target elementwise_pow_op [ 40%] Built target beam_search_decode_op [ 40%] Built target random_crop_op [ 40%] Built target sgd_op [ 40%] Built target fake_dequantize_op [ 40%] Built target pool_with_index_op [ 41%] Built target beam_search_op [ 42%] Built target unpooling [ 42%] Built target unpool_op [ 42%] Built target bilinear_interp_op [ 42%] Built target elementwise_add_op [ 42%] Built target get_places_op [ 42%] Built target slice_op [ 42%] Built target shrink_rnn_memory_op [ 42%] Built target adagrad_op [ 42%] Built target channel_close_op [ 42%] Built target fill_zeros_like_op [ 42%] Built target chunk_eval_op [ 42%] Built target clip_by_norm_op [ 42%] Built target bilinear_tensor_product_op [ 42%] Built target compare_op [ 42%] Built target multiplex_op [ 42%] Built target conditional_block_op [ 43%] Built target conv_shift_op [ 43%] Built target read_op [ 43%] Built target lstmp_op [ 43%] Built target crf_decoding_op [ 43%] Built target gru_unit_op [ 43%] Built target uniform_random_batch_size_like_op [ 43%] Built target proximal_adagrad_op [ 43%] Built target crop_op [ 43%] Built target reduce_prod_op [ 43%] Built target increment_op [ 43%] Built target delete_var_op [ 43%] Built target gather_op [ 43%] Built target elementwise_div_op [ 43%] Built target elementwise_min_op [ 44%] Built target elementwise_mul_op [ 44%] Built target fill_constant_op [ 44%] Built target squared_l2_norm_op [ 45%] Built target fill_op [ 45%] Built target ftrl_op [ 45%] Built target reshape_op [ 45%] Built target gaussian_random_op [ 45%] Built target reduce_min_op [ 45%] Built target fetch_op [ 45%] Built target go_op [ 45%] Built target matmul_op [ 45%] Built target hinge_loss_op [ 45%] Built target im2sequence_op [ 45%] Built target reader_op_registry [ 45%] Built target create_random_data_generator_op [ 45%] Built target open_files_op [ 45%] Built target create_multi_pass_reader_op [ 45%] Built target create_shuffle_reader_op [ 45%] Built target create_recordio_file_reader_op [ 45%] Built target create_batch_reader_op [ 46%] Built target create_threaded_reader_op [ 46%] Built target create_double_buffer_reader_op [ 47%] Built target create_custom_reader_op [ 48%] Built target iou_similarity_op [ 48%] Built target target_assign_op [ 48%] Built target polygon_box_transform_op [ 48%] Built target mine_hard_examples_op [ 48%] Built target multiclass_nms_op [ 48%] Built target prior_box_op [ 48%] Built target bipartite_match_op [ 48%] Built target box_coder_op [ 48%] Built target paddle_fluid_api [ 49%] Built target extern_threadpool [ 49%] Built target var_handle [ 49%] Built target op_handle_base [ 49%] Built target ssa_graph [ 50%] Built target ssa_graph_builder [ 51%] Built target computation_op_handle [ 51%] Built target ssa_graph_checker [ 51%] Built target fetch_op_handle [ 51%] Built target variable_visitor [ 51%] Built target broadcast_op_handle [ 52%] Built target rpc_op_handle [ 52%] Built target scale_loss_grad_op_handle [ 52%] Built target all_reduce_op_handle [ 52%] Built target ssa_graph_printer [ 52%] Built target reduce_op_handle [ 52%] Built target multi_devices_graph_builder [ 52%] Built target ssa_graph_builder_factory [ 52%] Built target ssa_graph_executor [ 52%] Built target scope_buffered_ssa_graph_executor [ 52%] Built target threaded_ssa_graph_executor [ 52%] Built target parallel_executor [ 53%] Built target prune [ 53%] Built target fuse_vars_op_handle [ 54%] Built target gather_op_handle [ 55%] Generating paddle_fluid.dir/paddle_fluid_api.objdir [ 55%] Generating paddle_fluid.dir/paddle_fluid_api.objdir [100%] Built target paddle_fluid [100%] Built target paddle_fluid_shared copying /home/daiwei/AnyQ/build/third_party/paddle/src/extern_paddle/paddle/fluid/inference/.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/inference copying /home/daiwei/AnyQ/build/third_party/install/paddle/paddle/fluid/inference/libpaddle_fluid. -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/inference [100%] Built target inference_lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/install/mklml/lib/libmklml_intel.so -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/install/mklml/lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/install/mklml/lib/libiomp5.so -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/install/mklml/lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/install/mklml/include -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/install/mklml [100%] Built target mklml_lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/paddle/fluid/pybind/pybind.h -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/paddle/fluid/pybind [100%] Built target pybind_lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/eigen3/src/extern_eigen3/Eigen/Core -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/eigen3/Eigen copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/eigen3/src/extern_eigen3/Eigen/src -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/eigen3/Eigen copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/eigen3/src/extern_eigen3/unsupported/Eigen -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/eigen3/unsupported [100%] Built target eigen3_lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/install/gflags/include -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/install/gflags copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/install/gflags/lib/libgflags.a -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/install/gflags/lib [100%] Built target gflags_lib copying /home/daiwei/AnyQ/build/third_party/install/paddle/third_party/boost/src/extern_boost/boost_1_41_0/boost -> /home/daiwei/AnyQ/build/third_party/install/paddle/fluid_install_dir/third_party/boost/ [100%] Built target boost_lib [100%] Built target inference_lib_dist [ 25%] Completed 'extern_paddle' [ 25%] Built target extern_paddle make: *** [all] Error 2

zhanghan1992 commented 6 years ago

rm -rf third_party/xgboost third_party/eigen make

daiw10 commented 6 years ago

谢谢 编译问题解决了，但是run server的时候会报以下错误

I0731 18:28:45.794803 23784 utils.cpp:590] RAW: in json_to_analysis_item, query:账号登录 I0731 18:28:45.794803 23784 utils.cpp:594] RAW: in json_to_analysis_item, type:(null) I0731 18:28:45.794803 23784 analysis_strategy.cpp:115] RAW: analysis_analysis size: 1 I0731 18:28:45.794803 23784 analysis_strategy.cpp:130] RAW: before use analysis strategy's analysis_result I0731 18:28:45.794803 23784 utils.cpp:609] RAW: query:账号登录 I0731 18:28:45.794803 23784 utils.cpp:611] RAW: tokens_basic size is 0 I0731 18:28:45.794803 23784 analysis_strategy.cpp:148] RAW: method_process method_wordseg start I0731 18:28:45.794803 23784 analysis_strategy.cpp:154] RAW: method_process method_wordseg sucess I0731 18:28:45.794803 23784 analysis_strategy.cpp:163] RAW: after use analysis strategy's analysis_result I0731 18:28:45.794803 23784 utils.cpp:609] RAW: query:账号登录 I0731 18:28:45.794803 23784 utils.cpp:611] RAW: tokens_basic size is 2 I0731 18:28:45.794803 23784 utils.cpp:614] RAW: tokens_basic buffer:账号 length:6 offset:0 analysis_term_weight:0.500000 I0731 18:28:45.794803 23784 utils.cpp:614] RAW: tokens_basic buffer:登录 length:6 offset:6 analysis_term_weight:0.500000 I0731 18:28:45.794803 23784 equal_solr_q_builder.cpp:48] RAW: equal solr_fetch_q=question:账号登录 I0731 18:28:45.794803 23784 term_retrieval.cpp:109] RAW: solr_fetch_q=question:账号登录 I0731 18:28:45.794803 23784 term_retrieval.cpp:119] RAW: url = http://127.0.0.1:8900/solr/collection1/select I0731 18:28:45.794803 23784 http_client.cpp:84] RAW: para_url: fl=id,question,answer&q=question%3A%E8%B4%A6%E5%8F%B7%E7%99%BB%E5%BD%95&rows=15&wt=json W0731 18:28:45.794803 23784 term_retrieval.cpp:157] RAW: solr_result[{"responseHeader":{"status":400,"QTime":1,"params":{"q":"question:账号登录","fl":"id,question,answer","rows":"15","wt":"json"}},"error":{"msg":"undefined field question","code":400}} ], response not exist; W0731 18:28:45.794803 23784 term_retrieval.cpp:212] RAW: solr_result_parse failed. E0731 18:28:45.794803 23784 retrieval_strategy.cpp:122] RAW: plugin term_recall_1 retrieval error E0731 18:28:45.794803 23784 anyq_strategy.cpp:69] RAW: retrieval module process error E0731 18:28:45.794803 23784 http_service_impl.cpp:133] RAW: anyq run_strategy failed!

yinweichong commented 6 years ago

需要按照教程README.md，获取anyq示例配置、启动solr，然后再运行run_server

# 获取anyq定制solr，anyq示例配置
cp ../tools/anyq_deps.sh .
sh anyq_deps.sh

# 启动solr, 依赖python-json, jdk>=1.8
cp ../tools/solr -rp solr_script
sh solr_script/anyq_solr.sh solr_script/sample_docs
daiw10 commented 6 years ago

这个流程跑了，好像是抛出异常了当时没注意，可能python2 转3 还需要修改些东西 Traceback (most recent call last): File "solr_script/solr_api.py", line 24, in solr_tools.set_engine_schema(sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5]) File "/home/daiwei/AnyQ/build/solr_script/solr_tools.py", line 114, in set_engine_schema response = urllib.request.urlopen(req, json.dumps(schema_config, ensure_ascii=False)) File "/opt/anaconda3/lib/python3.5/json/init.py", line 237, in dumps **kw).encode(obj) File "/opt/anaconda3/lib/python3.5/json/encoder.py", line 198, in encode chunks = self.iterencode(o, _one_shot=True) File "/opt/anaconda3/lib/python3.5/json/encoder.py", line 256, in iterencode return _iterencode(o, 0) TypeError: keys must be a string Traceback (most recent call last): File "solr_script/solr_api.py", line 30, in solr_tools.clear_documents(sys.argv[2], sys.argv[3], sys.argv[4]) File "/home/daiwei/AnyQ/build/solr_script/solr_tools.py", line 216, in clear_documents response = urllib.request.urlopen(req, urllib.parse.urlencode(params)) File "/opt/anaconda3/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/opt/anaconda3/lib/python3.5/urllib/request.py", line 464, in open req = meth(req) File "/opt/anaconda3/lib/python3.5/urllib/request.py", line 1183, in dorequest raise TypeError(msg) TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.

zhanghan1992 commented 6 years ago

运行部署脚本需要python 2.7