search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.74k stars 2.97k forks source link

Nvidia DPX2: run imagenet-console successfully ,but imagenet-camera gets errors #232

Closed Patrick-Woo closed 6 years ago

Patrick-Woo commented 6 years ago

I can run imagenet-console successfully on DPX2.

when I use C920 webcam and run imagenet-camera, it comes up with the errors below:

I guess I've already loaded the model and get the class result(in the terminal log). however, I cannot open the display window and see the result.

./imagenet-camera imagenet-camera args (1): 0 [./imagenet-camera]

[gstreamer] initialized gstreamer, version 1.8.2.0 [gstreamer] gstreamer decoder pipeline string: v4l2src device=/dev/video0 ! video/x-raw, width=(int)1920, height=(int)1080, format=RGB ! videoconvert ! video/x-raw, format=RGB ! videoconvert !appsink name=mysink

imagenet-camera: successfully initialized video device width: 1920 height: 1080 depth: 24 (bpp)

imageNet -- loading classification network model from: -- prototxt networks/googlenet.prototxt -- model networks/bvlc_googlenet.caffemodel -- class_labels networks/ilsvrc12_synset_words.txt -- input_blob 'data' -- output_blob 'prob' -- batch_size 2

[GIE] TensorRT version 3.0.2, build 3002 [GIE] attempting to open cache file networks/bvlc_googlenet.caffemodel.2.tensorcache [GIE] loading network profile from cache... networks/bvlc_googlenet.caffemodel.2.tensorcache nvrm_gpu: Bug 200215060 workaround enabled. [GIE] platform does not have FP16 support. [GIE] networks/bvlc_googlenet.caffemodel loaded [GIE] CUDA engine context initialized with 2 bindings [GIE] networks/bvlc_googlenet.caffemodel input binding index: 0 [GIE] networks/bvlc_googlenet.caffemodel input dims (b=2 c=3 h=224 w=224) size=1204224 [cuda] cudaAllocMapped 1204224 bytes, CPU 0x1053e0000 GPU 0x1053e0000 [GIE] networks/bvlc_googlenet.caffemodel output 0 prob binding index: 1 [GIE] networks/bvlc_googlenet.caffemodel output 0 prob dims (b=2 c=1000 h=1 w=1) size=8000 [cuda] cudaAllocMapped 8000 bytes, CPU 0x1055e0000 GPU 0x1055e0000 networks/bvlc_googlenet.caffemodel initialized. [GIE] networks/bvlc_googlenet.caffemodel loaded imageNet -- loaded 1000 class info entries networks/bvlc_googlenet.caffemodel initialized. default X screen 0: 3200 x 1080 [OpenGL] glDisplay display window initialized [OpenGL] creating 1920x1080 texture loaded image fontmapA.png (256 x 512) 2097152 bytes [cuda] cudaAllocMapped 2097152 bytes, CPU 0x1057e0000 GPU 0x1057e0000 [cuda] cudaAllocMapped 8192 bytes, CPU 0x1055e2000 GPU 0x1055e2000 [gstreamer] gstreamer transitioning pipeline to GST_STATE_PLAYING [gstreamer] gstreamer changed state from NULL to READY ==> mysink [gstreamer] gstreamer changed state from NULL to READY ==> videoconvert1 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1 [gstreamer] gstreamer changed state from NULL to READY ==> videoconvert0 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0 [gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0 [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0 [gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert1 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1 [gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0 [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer msg new-clock ==> pipeline0 [gstreamer] gstreamer msg stream-start ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0

imagenet-camera: camera open for streaming [gstreamer] gstreamer decoder onPreroll [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1059e0000 GPU 0x1059e0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x105fd0000 GPU 0x105fd0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1065c0000 GPU 0x1065c0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x106bb0000 GPU 0x106bb0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1071a0000 GPU 0x1071a0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x107790000 GPU 0x107790000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x107d80000 GPU 0x107d80000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108370000 GPU 0x108370000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108960000 GPU 0x108960000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108f50000 GPU 0x108f50000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x109540000 GPU 0x109540000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x109b30000 GPU 0x109b30000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10a120000 GPU 0x10a120000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10a710000 GPU 0x10a710000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10ad00000 GPU 0x10ad00000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10b2f0000 GPU 0x10b2f0000 [cuda] gstreamer camera -- allocated 16 ringbuffers, 6220800 bytes each [gstreamer] gstreamer changed state from READY to PAUSED ==> mysink [gstreamer] gstreamer msg async-done ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0 [cuda] gstreamer camera -- allocated 16 RGBA ringbuffers [gstreamer] gstreamer msg qos ==> v4l2src0 class 0526 - 0.107836 (desk) class 0527 - 0.125700 (desktop computer) class 0553 - 0.030763 (file, file cabinet, filing cabinet) class 0620 - 0.016277 (laptop, laptop computer) class 0632 - 0.011526 (loudspeaker, speaker, speaker unit, loudspeaker system, speaker system) class 0651 - 0.056109 (microwave, microwave oven) class 0664 - 0.244295 (monitor) class 0673 - 0.048897 (mouse, computer mouse) class 0713 - 0.010147 (photocopier) class 0745 - 0.014054 (projector) class 0760 - 0.019738 (refrigerator, icebox) class 0782 - 0.066791 (screen, CRT screen) class 0811 - 0.015403 (space heater) class 0851 - 0.012936 (television, television system) imagenet-camera: 24.42952% class #664 (monitor) [cuda] cudaGraphicsGLRegisterBuffer(&mInteropCUDA, mDMA, cudaGraphicsRegisterFlagsWriteDiscard) [cuda] operation not supported (error 71) (hex 0x47) [cuda] /home/nvidia/data/src/jetson-inference/util/display/glTexture.cpp:253 [cuda] cudaGetLastError() [cuda] operation not supported (error 71) (hex 0x47) [cuda] /home/nvidia/data/src/jetson-inference/util/cuda/cudaRGB.cu:62 [cuda] cudaRGBToRGBAf((uchar3)input, (float4)mRGBA[mLatestRGBA], mWidth, mHeight) [cuda] operation not supported (error 71) (hex 0x47)

dusty-nv commented 6 years ago

Hi Patrick, it looks like the part that is failing is the CUDA->OpenGL interopability that copies the camera video from CUDA into OpenGL texture. I don't have a DRIVE PX system so I am unable to debug, but if you take out the interop stuff and copy it through the CPU it should work..

-------- Original message -------- From: Patrick-Woo notifications@github.com Date: 5/15/18 8:14 AM (GMT-05:00) To: dusty-nv/jetson-inference jetson-inference@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [dusty-nv/jetson-inference] Nvidia DPX2: run imagenet-console successfully ,but imagenet-camera gets errors (#232)

I can run imagenet-console successfully on DPX2.

when I use C920 webcam and run imagenet-camera, it comes up with the errors below:

I guess I've already loaded the model and get the class result(in the terminal log). however, I cannot open the display window and see the result.

./imagenet-camera imagenet-camera args (1): 0 [./imagenet-camera]

[gstreamer] initialized gstreamer, version 1.8.2.0 [gstreamer] gstreamer decoder pipeline string: v4l2src device=/dev/video0 ! video/x-raw, width=(int)1920, height=(int)1080, format=RGB ! videoconvert ! video/x-raw, format=RGB ! videoconvert !appsink name=mysink

imagenet-camera: successfully initialized video device width: 1920 height: 1080 depth: 24 (bpp)

imageNet -- loading classification network model from: -- prototxt networks/googlenet.prototxt -- model networks/bvlc_googlenet.caffemodel -- class_labels networks/ilsvrc12_synset_words.txt -- input_blob 'data' -- output_blob 'prob' -- batch_size 2

[GIE] TensorRT version 3.0.2, build 3002 [GIE] attempting to open cache file networks/bvlc_googlenet.caffemodel.2.tensorcache [GIE] loading network profile from cache... networks/bvlc_googlenet.caffemodel.2.tensorcache nvrm_gpu: Bug 200215060 workaround enabled. [GIE] platform does not have FP16 support. [GIE] networks/bvlc_googlenet.caffemodel loaded [GIE] CUDA engine context initialized with 2 bindings [GIE] networks/bvlc_googlenet.caffemodel input binding index: 0 [GIE] networks/bvlc_googlenet.caffemodel input dims (b=2 c=3 h=224 w=224) size=1204224 [cuda] cudaAllocMapped 1204224 bytes, CPU 0x1053e0000 GPU 0x1053e0000 [GIE] networks/bvlc_googlenet.caffemodel output 0 prob binding index: 1 [GIE] networks/bvlc_googlenet.caffemodel output 0 prob dims (b=2 c=1000 h=1 w=1) size=8000 [cuda] cudaAllocMapped 8000 bytes, CPU 0x1055e0000 GPU 0x1055e0000 networks/bvlc_googlenet.caffemodel initialized. [GIE] networks/bvlc_googlenet.caffemodel loaded imageNet -- loaded 1000 class info entries networks/bvlc_googlenet.caffemodel initialized. default X screen 0: 3200 x 1080 [OpenGL] glDisplay display window initialized [OpenGL] creating 1920x1080 texture loaded image fontmapA.png (256 x 512) 2097152 bytes [cuda] cudaAllocMapped 2097152 bytes, CPU 0x1057e0000 GPU 0x1057e0000 [cuda] cudaAllocMapped 8192 bytes, CPU 0x1055e2000 GPU 0x1055e2000 [gstreamer] gstreamer transitioning pipeline to GST_STATE_PLAYING [gstreamer] gstreamer changed state from NULL to READY ==> mysink [gstreamer] gstreamer changed state from NULL to READY ==> videoconvert1 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1 [gstreamer] gstreamer changed state from NULL to READY ==> videoconvert0 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0 [gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0 [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0 [gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert1 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1 [gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0 [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer msg new-clock ==> pipeline0 [gstreamer] gstreamer msg stream-start ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0

imagenet-camera: camera open for streaming [gstreamer] gstreamer decoder onPreroll [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1059e0000 GPU 0x1059e0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x105fd0000 GPU 0x105fd0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1065c0000 GPU 0x1065c0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x106bb0000 GPU 0x106bb0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x1071a0000 GPU 0x1071a0000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x107790000 GPU 0x107790000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x107d80000 GPU 0x107d80000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108370000 GPU 0x108370000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108960000 GPU 0x108960000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x108f50000 GPU 0x108f50000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x109540000 GPU 0x109540000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x109b30000 GPU 0x109b30000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10a120000 GPU 0x10a120000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10a710000 GPU 0x10a710000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10ad00000 GPU 0x10ad00000 [cuda] cudaAllocMapped 6220800 bytes, CPU 0x10b2f0000 GPU 0x10b2f0000 [cuda] gstreamer camera -- allocated 16 ringbuffers, 6220800 bytes each [gstreamer] gstreamer changed state from READY to PAUSED ==> mysink [gstreamer] gstreamer msg async-done ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0 [cuda] gstreamer camera -- allocated 16 RGBA ringbuffers [gstreamer] gstreamer msg qos ==> v4l2src0 class 0526 - 0.107836 (desk) class 0527 - 0.125700 (desktop computer) class 0553 - 0.030763 (file, file cabinet, filing cabinet) class 0620 - 0.016277 (laptop, laptop computer) class 0632 - 0.011526 (loudspeaker, speaker, speaker unit, loudspeaker system, speaker system) class 0651 - 0.056109 (microwave, microwave oven) class 0664 - 0.244295 (monitor) class 0673 - 0.048897 (mouse, computer mouse) class 0713 - 0.010147 (photocopier) class 0745 - 0.014054 (projector) class 0760 - 0.019738 (refrigerator, icebox) class 0782 - 0.066791 (screen, CRT screen) class 0811 - 0.015403 (space heater) class 0851 - 0.012936 (television, television system) imagenet-camera: 24.42952% class #664 (monitor) [cuda] cudaGraphicsGLRegisterBuffer(&mInteropCUDA, mDMA, cudaGraphicsRegisterFlagsWriteDiscard) [cuda] operation not supported (error 71) (hex 0x47) [cuda] /home/nvidia/data/src/jetson-inference/util/display/glTexture.cpp:253 [cuda] cudaGetLastError() [cuda] operation not supported (error 71) (hex 0x47) [cuda] /home/nvidia/data/src/jetson-inference/util/cuda/cudaRGB.cu:62 [cuda] cudaRGBToRGBAf((uchar3)input, (float4)mRGBA[mLatestRGBA], mWidth, mHeight) [cuda] operation not supported (error 71) (hex 0x47)

- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-inference/issues/232, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AOpDK-zYT3x3MgrNoBXoI5MWPvqcxibQks5tysZ4gaJpZM4T_bet.

This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

Patrick-Woo commented 6 years ago

thank you for your prompt reply. could you please tell me how to use cpu to copy the video out? I am less familiar with cpp and only have experence with caffe modeling. thank you in advance.

dusty-nv commented 6 years ago

You should be able to use the glTexture::UploadCPU() function to do it: https://github.com/dusty-nv/jetson-inference/blob/e168b746d540fd739a6fa6ba652b5a5e8c6ffa08/util/display/glTexture.h#L53

Pass in the pointer of the buffer that you want to end up in the texture.

From: Patrick-Woo notifications@github.com Sent: Tuesday, May 15, 2018 8:14 PM To: dusty-nv/jetson-inference jetson-inference@noreply.github.com Cc: Dustin Franklin dustinf@nvidia.com; Comment comment@noreply.github.com Subject: Re: [dusty-nv/jetson-inference] Nvidia DPX2: run imagenet-console successfully ,but imagenet-camera gets errors (#232)

thank you for your prompt reply. could you please tell me how to use cpu to copy the video out? I am less familiar with cpp and only have experence with caffe modeling. thank you in advance.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-inference/issues/232#issuecomment-389354253, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AOpDK7hySz1bA3Ppsjt2K2PjreIcxaiVks5ty287gaJpZM4T_bet.

This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

Patrick-Woo commented 6 years ago

@dusty-nv thank you for your reply and I am still confused about how to "Pass in the pointer of the buffer that you want to end up in the texture."

I could see there is a uploadCPU function. but where should I use this function?could you pls give me some sample code?

I have to show my leader PX2 can be used to do classification before the end of this week. it is quit difficult for me(less cpp knowledge) to understand your source code in such a short time. So, if you could provide me with the code, I will be really really appreciated.

Patrick-Woo commented 6 years ago

@dusty-nv please do me a faviour and present me some sample code. thank you in advance.

dusty-nv commented 6 years ago

Comment out lines 197-207 of imagenet-camera.cpp: https://github.com/dusty-nv/jetson-inference/blob/e168b746d540fd739a6fa6ba652b5a5e8c6ffa08/imagenet-camera/imagenet-camera.cpp#L197

and replace with the following:

/*void* tex_map = texture->MapCUDA();

if( tex_map != NULL )
{
    cudaMemcpy(tex_map, imgRGBA, texture->GetSize(), cudaMemcpyDeviceToDevice);
    texture->Unmap();
}*/

CUDA(cudaDeviceSynchronize());
texture->UploadCPU(imageRGBA);

The performance hit may not be very good from not using GL interop. I am not sure why the interop isn't work on DRIVE PX, perhaps it is related to multiple GPUs.

Patrick-Woo commented 6 years ago

@dusty-nv Thank you for your time. I am really really appreciated.

However, after I replace the code you offered and compile it successfully, a new error message occurs when I run imagenet_camera in the bin folder.

source code: CUDA(cudaDeviceSynchronize()); texture->UploadCPU(imageRGBA); //imageRGBA should be imgRGBA I guess

error message: [cuda] gstreamer camera -- allocated 16 ringbuffers, 2764800 bytes each [gstreamer] gstreamer changed state from READY to PAUSED ==> mysink [gstreamer] gstreamer msg async-done ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0 [cuda] gstreamer camera -- allocated 16 RGBA ringbuffers class 0410 - 0.014961 (apiary, bee house) class 0446 - 0.025046 (binder, ring-binder) class 0478 - 0.047681 (carton) class 0526 - 0.015736 (desk) class 0527 - 0.027656 (desktop computer) class 0549 - 0.040431 (envelope) class 0553 - 0.024300 (file, file cabinet, filing cabinet) class 0664 - 0.011072 (monitor) class 0681 - 0.014861 (notebook, notebook computer) class 0713 - 0.185375 (photocopier) class 0742 - 0.252791 (printer) class 0745 - 0.043518 (projector) class 0760 - 0.048958 (refrigerator, icebox) class 0782 - 0.018003 (screen, CRT screen) class 0922 - 0.015056 (menu) imagenet-camera: 25.27907% class #742 (printer) Segmentation fault (core dumped)

dusty-nv commented 6 years ago

@Patrick-Woo please change imagenet-camera.cpp:154 to reflect the following:

if( !camera->ConvertRGBA(imgCUDA, &imgRGBA, true) )

This will make sure that imgRGBA is accessible from CPU. Sorry for forgetting about that detail earlier.

Patrick-Woo commented 6 years ago

It really works. Thank you so much.

without your help, I could not finish my job before the deadline. Again, thank you very much.