Open xsscss opened 5 years ago
实验环境:python3.5 paddlepaddle-gpu==1.0.0.post87 ubuntu16.04 GPU 使用 两个1080Ti 执行 sh run.sh --train --pass_num 10 时, 在执行到2018-12-17 11:15:38,728 - brc - INFO - Training the model... 时 报错如下
sh run.sh --train --pass_num 10
2018-12-17 11:15:38,728 - brc - INFO - Training the model...
Traceback (most recent call last): File "run.py", line 636, in <module> train(logger, args) File "run.py", line 422, in train return_numpy=False) File "/home/user/.local/lib/python3.5/site-packages/paddle/fluid/parallel_executor.py", line 260, in run self.executor.run(fetch_list, fetch_var_name) paddle.fluid.core.EnforceNotMet: Variable must be type N6paddle9framework9LoDTensorE, the holding type is N6paddle9framework12SelectedRowsE at [/paddle/paddle/fluid/framework/variable.h:33] PaddlePaddle Call Stacks: 0 0x7eff7151c486p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 486 1 0x7eff715fbe6cp paddle::framework::LoDTensor const& paddle::framework::Variable::Get<paddle::framework::LoDTensor>() const + 300 2 0x7eff72534e27p paddle::operators::SumOp::GetExpectedKernelType(paddle::framework::ExecutionContext const&) const + 455 3 0x7eff727a0127p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 199 4 0x7eff7279d3fcp paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 252 5 0x7eff725de157p 6 0x7eff725fa780p 7 0x7eff725f9fe5p paddle::framework::details::OpHandleBase::RunAndRecordEvent(std::function<void ()> const&) + 805 8 0x7eff725ddc2fp paddle::framework::details::ComputationOpHandle::RunImpl() + 95 9 0x7eff725fb085p paddle::framework::details::OpHandleBase::Run(bool) + 117 10 0x7eff725a9efap 11 0x7eff715eaa03p std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&) + 35 12 0x7eff715ea1d7p std::__future_base::_State_base::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&, bool&) + 39 13 0x7effe9501a99p 14 0x7eff725a8ed2p 15 0x7eff715ec1f4p ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const + 404 16 0x7effe41b7c80p 17 0x7effe94fa6bap 18 0x7effe923041dp clone + 109
请更新到fluid1.2版本,fluid1.2有修复这个问题,谢谢
实验环境:python3.5 paddlepaddle-gpu==1.0.0.post87 ubuntu16.04 GPU 使用 两个1080Ti 执行
sh run.sh --train --pass_num 10
时, 在执行到2018-12-17 11:15:38,728 - brc - INFO - Training the model...
时 报错如下