search

dusty-nv / jetson-utils

C++/CUDA/Python multimedia utilities for NVIDIA Jetson
MIT License
709 stars 284 forks source link

[gstreamer] gstDecoder -- Could not demultiplex stream. #33

Open BullotLo opened 4 years ago

BullotLo commented 4 years ago

Hi, I am trying to use detectnet to use the model I trained and run inference on my jetson nano. I need to get the video stream from a file located on an external hard drive so I did clone the dev branch and I'm trying to make the detectnet example work. My video is an .avi file with the mjpg codec.

I have an issue when I run the following command:

jetson@Jetson:~/jetson-inference/build/aarch64/bin$ detectnet --prototxt=$NET/deploy.prototxt --model=$NET/snapshot_iter_52000.caffemodel --input_blob=data --output-cvg=coverage --output-bbox=bboxes /media/5C71-CE67/TB/Video/Mouse_short.avi --input-codec=mjpeg display://

I get two errors: the gstDecoder cannot demultiplex the stream and the plugin creator FlattnContact_TRT does not register. I don't really understand what that mean but the result is that the frame don't get extracted.

My terminal:

[gstreamer] initialized gstreamer, version 1.14.5.0 [gstreamer] gstDecoder -- creating decoder for /media/5C71-CE67/TB/Video/Mouse_short.avi [gstreamer] gstDecoder -- Could not demultiplex stream. [gstreamer] gstDecoder -- pipeline string: [gstreamer] filesrc location=/media/5C71-CE67/TB/Video/Mouse_short.avi ! avidemux ! queue ! nvjpegdec ! video/x-raw ! appsink name=mysink [video] created gstDecoder from file:///media/5C71-CE67/TB/Video/Mouse_short.avi

gstDecoder video options:

-- URI: file:///media/5C71-CE67/TB/Video/Mouse_short.avi

URI -- using default display device 0 [OpenGL] glDisplay -- X screen 0 resolution: 1920x1080 [OpenGL] glDisplay -- X window resolution: 1920x1080 [OpenGL] glDisplay -- display device initialized (1920x1080) [video] created glDisplay from display://

glDisplay video options:

-- URI: display://

detectNet -- loading detection network model from: -- prototxt /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/deploy.prototxt -- model /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/snapshot_iter_52000.caffemodel -- input_blob 'data' -- output_cvg 'coverage' -- output_bbox 'bboxes' -- mean_pixel 0.000000 -- mean_binary NULL -- class_labels NULL -- threshold 0.500000 -- batch_size 1

[TRT] TensorRT version 7.1.0 [TRT] loading NVIDIA plugins... [TRT] Plugin creator registration succeeded - ::GridAnchor_TRT [TRT] Plugin creator registration succeeded - ::NMS_TRT [TRT] Plugin creator registration succeeded - ::Reorg_TRT [TRT] Plugin creator registration succeeded - ::Region_TRT [TRT] Plugin creator registration succeeded - ::Clip_TRT [TRT] Plugin creator registration succeeded - ::LReLU_TRT [TRT] Plugin creator registration succeeded - ::PriorBox_TRT [TRT] Plugin creator registration succeeded - ::Normalize_TRT [TRT] Plugin creator registration succeeded - ::RPROI_TRT [TRT] Plugin creator registration succeeded - ::BatchedNMS_TRT [TRT] Could not register plugin creator: ::FlattenConcat_TRT [TRT] Plugin creator registration succeeded - ::CropAndResize [TRT] Plugin creator registration succeeded - ::DetectionLayer_TRT [TRT] Plugin creator registration succeeded - ::Proposal [TRT] Plugin creator registration succeeded - ::ProposalLayer_TRT [TRT] Plugin creator registration succeeded - ::PyramidROIAlign_TRT [TRT] Plugin creator registration succeeded - ::ResizeNearest_TRT [TRT] Plugin creator registration succeeded - ::Split [TRT] Plugin creator registration succeeded - ::SpecialSlice_TRT [TRT] Plugin creator registration succeeded - ::InstanceNormalization_TRT [TRT] detected model format - caffe (extension '.caffemodel') [TRT] desired precision specified for GPU: FASTEST [TRT] requested fasted precision for device GPU without providing valid calibrator, disabling INT8 [TRT] native precisions detected for GPU: FP32, FP16 [TRT] selecting fastest native precision for GPU: FP16 [TRT] attempting to open engine cache file /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/snapshot_iter_52000.caffemodel.1.1.7100.GPU.FP16.engine [TRT] loading network plan from engine cache... /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/snapshot_iter_52000.caffemodel.1.1.7100.GPU.FP16.engine [TRT] device GPU, loaded /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/snapshot_iter_52000.caffemodel [TRT] Deserialize required 5707631 microseconds. [TRT]
[TRT] CUDA engine context initialized on device GPU: [TRT] -- layers 69 [TRT] -- maxBatchSize 1 [TRT] -- workspace 0 [TRT] -- deviceMemory 47227904 [TRT] -- bindings 3 [TRT] binding 0 -- index 0 -- name 'data' -- type FP32 -- in/out INPUT -- # dims 3 -- dim #0 3 (SPATIAL) -- dim #1 480 (SPATIAL) -- dim #2 752 (SPATIAL) [TRT] binding 1 -- index 1 -- name 'coverage' -- type FP32 -- in/out OUTPUT -- # dims 3 -- dim #0 1 (SPATIAL) -- dim #1 30 (SPATIAL) -- dim #2 47 (SPATIAL) [TRT] binding 2 -- index 2 -- name 'bboxes' -- type FP32 -- in/out OUTPUT -- # dims 3 -- dim #0 4 (SPATIAL) -- dim #1 30 (SPATIAL) -- dim #2 47 (SPATIAL) [TRT]
[TRT] binding to input 0 data binding index: 0 [TRT] binding to input 0 data dims (b=1 c=3 h=480 w=752) size=4331520 [TRT] binding to output 0 coverage binding index: 1 [TRT] binding to output 0 coverage dims (b=1 c=1 h=30 w=47) size=5640 [TRT] binding to output 1 bboxes binding index: 2 [TRT] binding to output 1 bboxes dims (b=1 c=4 h=30 w=47) size=22560 [TRT]
[TRT] device GPU, /home/jetson/jetson-inference/build/aarch64/bin/networks/Mouse_780_1000ep_nosub/snapshot_iter_52000.caffemodel initialized. [TRT] detectNet -- number object classes: 1 [TRT] detectNet -- maximum bounding boxes: 1410 [gstreamer] opening gstDecoder for streaming, transitioning pipeline to GST_STATE_PLAYING [gstreamer] gstreamer changed state from NULL to READY ==> mysink [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0 [gstreamer] gstreamer changed state from NULL to READY ==> nvjpegdec0 [gstreamer] gstreamer changed state from NULL to READY ==> queue0 [gstreamer] gstreamer changed state from NULL to READY ==> avidemux1 [gstreamer] gstreamer changed state from NULL to READY ==> filesrc0 [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0 [gstreamer] gstreamer changed state from READY to PAUSED ==> nvjpegdec0 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> queue0 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer stream status CREATE ==> sink [gstreamer] gstreamer changed state from READY to PAUSED ==> avidemux1 [gstreamer] gstreamer changed state from READY to PAUSED ==> filesrc0 [gstreamer] gstreamer stream status ENTER ==> sink [gstreamer] gstreamer avidemux1 ERROR Could not demultiplex stream. [gstreamer] gstreamer Debugging info: gstavidemux.c(4413): gst_avi_demux_stream_header_pull (): /GstPipeline:pipeline0/GstAviDemux:avidemux1: pull_range flow reading header: eos detectnet: failed to capture video frame detectnet: failed to capture video frame detectnet: failed to capture video frame .......

dusty-nv commented 4 years ago

the plugin creator FlattnContact_TRT does not register

You can ignore that message.

Can you try converting your AVI file to MKV or MP4 with H.264 encoding using ffmpeg or another tool?

BullotLo commented 4 years ago

Thanks for the answer I converted my file with the ffmpeg as you suggested: $ ffmpeg -i Mouse_short.avi -c:v h264 Mouse_short.mp4

Now when I try to run with this command: $ detectnet --prototxt=$NET/deploy.prototxt --model=$NET/snapshot_iter_52000.caffemodel --input_blob=data --output-cvg=coverage --output-bbox=bboxes /media/5C71-CE67/TB/Video/Mouse_short.mp4 --input-codec=h264 display://

The terminal show this:

[gstreamer] initialized gstreamer, version 1.14.5.0 [gstreamer] gstDecoder -- creating decoder for /media/5C71-CE67/TB/Video/Mouse_short.mp4 Opening in BLOCKING MODE NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 NVMEDIA: NVMEDIABufferProcessing: 1504: NvMediaParserParse Unsupported Codec

and then block himself. Even when I try to quit the process that keep running with Ctrl+C he show me this line but the process is not killed:

^C[NvMediaParserParse:357] Video parser parse failed: 0received SIGINT

I have to close the terminal to kill the process

BullotLo commented 4 years ago

I saw that the master branch was updated recently with the new detectnet file so I pulled this one and tried to make things simpler by running the example already in the git "my-detection.py" with the default model ssd-mobilenet-v2. I just changer the line:

camera = jetson.utils.videoSource("file:///media/5C71-CE67/TB/Video/Mouse_short.mkv")

I also tried with the MKV format but the problem stay the same.

dusty-nv commented 4 years ago

Hmmm - would you be able to upload the video somewhere (like google drive or something) so that I could try it?

BullotLo commented 4 years ago

Yes I can do that, the videos in AVI, MP4 and MKV format can be found here:

https://drive.google.com/file/d/1Wj0v5Gp3tFncCBqEKsDT3iYqw4tZVGW-/view?usp=sharing

thanks for the help