How to Processes the video Frame Fastly in Jetson Tx2?

[sathees07]

Hi @nasiy how to set on every fifth video frame to be processed instead of having all the frames per second to be processed in movie input?

[naisy]

Hi @sathees07, I think that it changes depending on whether 5 frames is a specification or an experimental value.

1. If you think by frame, you have to start to edit webcam.py. You first need to understand the difference between the implementation of webcam/streaming and movie frame reading. (issues #52) Next, you can read the frame until frame_counter == 5 after object detection and draw the previous box.

2. If you think by interval seconds, you can read the frame for interval seconds after object detection and draw the previous box.

Each has its advantages and disadvantages. Advantage of 1: It is surely processed in frame units. It will be easier to calculate the load on TX2. Disadvantage of 1: In webcam/streaming, you have to introduce slow frame read. This is the same frame read method as lib/video.py. Advantage of 2: As long as TX2 resources are not insufficient, it is processed in interval seconds. It does not depend on the frame rate, so it is a very powerful parameter adjustment in real-time demo using webcam/streaming. Disadvantage of 2: The number of frames for which detection is updated will be uneven. If the input src is a movie file, this will not be possible because the reading process of all the frames will be finished in an instant.

I think the implementation will change depending on what you want to do. I implemented method 2 + tracking, but did not add this update because multiple tracking is too slow.

[sathees07]

Thanks @naisy I am trying to do mask rcnn on Jetson Tx2. For image it process 1FPS. For video it works slowly.. Is any way to improve this.

[naisy]

Then how about next methods?

1. edit movie file and down frame rate. or
2. edit lib/video.py and add frame skip into read() method.

before

        self.ret, self.frame = self.vid.read()

after

        for i in range(5):
            self.ret, self.frame = self.vid.read()

[sathees07]

i tried this but it process slowly. How could i speed the detection for mask rcnn. What is te process behind work thread? i set worker thread to 4 in jetson tx2 but it does not work, it works only on 1 worker thread. And I want to process(object detection) every 20th frame of the video.

[naisy]

worker_threads is for processing multiple frames simultaneously. In TX2, there is no enough resources. It can not be set many.

If you edit lib/video.py to skip 20 frames and you feel slow, it won't go any faster.

after

        for i in range(20):
            self.ret, self.frame = self.vid.read()

[sathees07]

thanks for your response. i dont want to process full video i only need to process 20th frame. i want to skip the remaining frames and i only want to process every 20th frame.how to do?

for i in range(20): self.ret, self.frame = self.vid.read() it not takes 20th it process every frame

[sathees07]

It works!!! Thanks @naisy ..... It's my fault. You did an awesome work.

[sathees07]

hi @naisy i have trained a model using ssd mobilenet, it works fine in my laptop but when i tried that on jetson tx2 this error occurs.

Traceback (most recent call last): File "/home/nvidia/.local/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 418, in import_graph_def graph._c_graph, serialized, options) # pylint: disable=protected-access tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: ToFloat = CastDstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false. (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).

Help me to solve this error.

[naisy]

Hi @sathees07,

Perhaps, tensorflow version is old. Please check laptop and TX2 tensorflow version.

[sathees07]

yeah @naisy, i tried that. thank you