Awesome video understanding toolkits based on PaddlePaddle. It supports video data annotation tools, lightweight RGB and skeleton based action recognition model, practical applications for video tagging and sport action detection.
Apache License 2.0
1.53k
stars
380
forks
source link
Out of memory error on GPU 0. Cannot allocate 63.000244MB memory on GPU 0, 1.957780GB memory has been allocated and available memory is only 43.232811MB. #419
错误信息:
W0426 23:22:43.432502 11372 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 5.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0426 23:22:43.447456 11372 device_context.cc:465] device: 0, cuDNN Version: 7.6.
[04/26 23:22:47] ppdet.utils.checkpoint INFO: Finish loading model weights: G:/work_py/notebook/paddle/SlowFast/work/faster_rcnn_r50_fpn_1x_coco.pdparams
Performing Human Detection for each frame
Traceback (most recent call last):
File "tools/predict.py", line 276, in
main()
File "tools/predict.py", line 179, in main
inputs = InferenceHelper.preprocess(file)
File "G:\work_py\notebook\paddle\PaddleVideo\PaddleVideo-develop\tools\utils.py", line 1315, in preprocess
detection_txt_list = detection_inference(selected_frame_list,
File "G:\work_py\notebook\paddle\PaddleVideo\PaddleVideo-develop\tools\ava_predict.py", line 281, in detection_inference
detection_trainer.predict(frame_paths, output_dir=output_dir, save_txt=True)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\engine\trainer.py", line 517, in predict
outs = self.model(data)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in call
return self._dygraph_call_func(*inputs, kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
outputs = self.forward(*inputs, *kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\meta_arch.py", line 56, in forward
out = self.get_pred()
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\faster_rcnn.py", line 104, in get_pred
bbox_pred, bbox_num = self._forward()
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\faster_rcnn.py", line 74, in _forward
body_feats = self.neck(body_feats)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in call
return self._dygraph_call_func(inputs, kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
outputs = self.forward(*inputs, *kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\necks\fpn.py", line 193, in forward
upsample = F.interpolate(
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\nn\functional\common.py", line 467, in interpolate
out = _C_ops.nearest_interp_v2(x, dy_attr)
SystemError: (Fatal) Operator nearest_interp_v2 raises an struct paddle::memory::allocation::BadAlloc exception.
The exception content is
:ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 63.000244MB memory on GPU 0, 1.958673GB memory has been allocated and available memory is only 42.318748MB.
Please check whether there is any other process using GPU 0.
If yes, please stop them, or start PaddlePaddle on another GPU.
If no, please decrease the batch size of your model.
Describe the bug 在win10,gpu为2gb,PaddleVideo-develop2.1.0上,想通过使用cpu运行预测推理,结果执行抛出gpu 我执行的命令为: python tools/predict.py -c G:/work_py/notebook/paddle/SlowFast/work/abnoraml_action.yaml --input_file "G:/work_py/notebook/paddle/SlowFast/work/data/wave_9.mp4" --model_file "G:/work_py/notebook/paddle/SlowFast/work/inference_output/AVA_SlowFast_FastRcnn.pdmodel" --params_file "G:/work_py/notebook/paddle/SlowFast/work/inference_output/AVA_SlowFast_FastRcnn.pdiparams" --use_gpu=False --use_tensorrt=False 其中predict.py文件中加入os.environ['CUDA_VISIBLE_DEVICES'] = '-1' 也不起作用
错误信息: W0426 23:22:43.432502 11372 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 5.0, Driver API Version: 10.1, Runtime API Version: 10.1 W0426 23:22:43.447456 11372 device_context.cc:465] device: 0, cuDNN Version: 7.6. [04/26 23:22:47] ppdet.utils.checkpoint INFO: Finish loading model weights: G:/work_py/notebook/paddle/SlowFast/work/faster_rcnn_r50_fpn_1x_coco.pdparams Performing Human Detection for each frame Traceback (most recent call last): File "tools/predict.py", line 276, in
main()
File "tools/predict.py", line 179, in main
inputs = InferenceHelper.preprocess(file)
File "G:\work_py\notebook\paddle\PaddleVideo\PaddleVideo-develop\tools\utils.py", line 1315, in preprocess
detection_txt_list = detection_inference(selected_frame_list,
File "G:\work_py\notebook\paddle\PaddleVideo\PaddleVideo-develop\tools\ava_predict.py", line 281, in detection_inference
detection_trainer.predict(frame_paths, output_dir=output_dir, save_txt=True)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\engine\trainer.py", line 517, in predict
outs = self.model(data)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in call
return self._dygraph_call_func(*inputs, kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
outputs = self.forward(*inputs, *kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\meta_arch.py", line 56, in forward
out = self.get_pred()
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\faster_rcnn.py", line 104, in get_pred
bbox_pred, bbox_num = self._forward()
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\architectures\faster_rcnn.py", line 74, in _forward
body_feats = self.neck(body_feats)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in call
return self._dygraph_call_func(inputs, kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
outputs = self.forward(*inputs, *kwargs)
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddledet-2.3.0-py3.8.egg\ppdet\modeling\necks\fpn.py", line 193, in forward
upsample = F.interpolate(
File "E:\pfile\anaconda3\envs\paddle\lib\site-packages\paddle\nn\functional\common.py", line 467, in interpolate
out = _C_ops.nearest_interp_v2(x, dy_attr)
SystemError: (Fatal) Operator nearest_interp_v2 raises an struct paddle::memory::allocation::BadAlloc exception.
The exception content is
:ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 63.000244MB memory on GPU 0, 1.958673GB memory has been allocated and available memory is only 42.318748MB.
Please check whether there is any other process using GPU 0.
If no, please decrease the batch size of your model.
(at ..\paddle\fluid\memory\allocation\cuda_allocator.cc:79) . (at ..\paddle\fluid\imperative\tracer.cc:221)