search

OlafenwaMoses / ImageAI

A python library built to empower developers to build applications and systems with self-contained Computer Vision capabilities
https://www.genxr.co/#products
MIT License
8.55k stars 2.19k forks source link

Parameters "frames_per_second" and "frame_detection_interval" do not work correctly #235

Closed wangjin2000 closed 5 years ago

wangjin2000 commented 5 years ago

When trying to use above two parameters for object detection, I found their behavior are very strange. For example, when I load a video that has following features: fps = 23.976023976023978 number of frames = 3600 duration (S) = 150.14999999999998 duration (M:S) = 2:30.149999999999977

Then I applied a custom object detection analysis as below: detector = VideoObjectDetection() detector.setModelTypeAsYOLOv3() detector.setModelPath(os.path.join(model_path, model_file)) detector.loadModel()

custom = detector.CustomObjects(person=True) detector.detectCustomObjectsFromVideo(custom_objects=custom, input_file_path=os.path.join(video_path, video_file), output_file_path=os.path.join(video_path, "greenbook_analysis_second") ,
frames_per_second=24, frame_detection_interval = 12, per_frame_function=forSecond, minimum_percentage_probability=95)

First of all, I am not sure why we need to manually set parameter "frames_per_second" here. At very beginning, I thought it is for purpose of reducing fps of output video, say, change original 24 fps to 12 fps, but actually, if you set to 12, all 3600 frames are still saved but the total time mark doubled. So I finally set frames_per_second to original 24 to keep the same time length.

Secondly, the parameter "frame_detection_interval" does not take effect. I.e., it still goes though every frame.

Followings are output of above function call. From the output we can see that there are 24 elements in output object array for each second. Since we used the "frame_detection_interval = 12" should we only get 2 elements for each second?

More interesting thing is: if we set "frames_per_second=12", the output array will only have 12 elements for each second.

Please let me know if I misused these two parameters.

Thanks

output if： frames_per_second=24, frame_detection_interval = 12 ......................................... SECOND : 2 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], [], []] Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] Output average count for unique objects in the last second: {} ------------END OF A SECOND -------------- SECOND : 3 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.89290833473206, 'box_points': (442, 86, 1053, 686)}], [{'name': 'person', 'percentage_probability': 99.86517429351807, 'box_points': (440, 82, 1053, 681)}]] Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}, {'person': 1}] Output average count for unique objects in the last second: {'person': 0.5416666666666666} ------------END OF A SECOND --------------

output if： frames_per_second=12, frame_detection_interval = 12 ..................................................... SECOND : 2 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], []] (Note: Only 12 elements!) Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] Output average count for unique objects in the last second: {} ------------END OF A SECOND -------------- SECOND : 3 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], []] Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] Output average count for unique objects in the last second: {} ------------END OF A SECOND -------------- SECOND : 4 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], []] Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] Output average count for unique objects in the last second: {} ------------END OF A SECOND -------------- SECOND : 5 Array for the outputs of each frame [[], [], [], [], [], [], [], [], [], [], [], [{'percentage_probability': 99.89290833473206, 'name': 'person', 'box_points': (442, 86, 1053, 686)}]] Array for output count for unique objects in each frame : [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {'person': 1}] Output average count for unique objects in the last second: {'person': 0.08333333333333333} ------------END OF A SECOND --------------

OlafenwaMoses commented 5 years ago

Thanks for the comment. Let me address the issues you raised.

1) As per the frames_per_second paramater, the ImageAI API was engineered to allow users to customize the FPS for their output videos. 2) The frame_detection_interval only skips the number of frames specified during detection. However, all the frames are saved and the data from the detection of the previous frame is still persistently augmented on the subsequent (non-detected) frames.

wangjin2000 commented 5 years ago

Thanks for your quick response. Based on your explanation, I think it is tricky when setting these two parameters at the same time. For example, if the original FPS is 24, output FPS is to be 12 (BTW, there is no way to set FPS greater than original FPS so a check is needed in codes), how does the program work if the "frame_detection_interval" is set to 5? I guess use a parameter "time_detection_interval" makes more sense and easy implemented?

Thanks.

OlafenwaMoses commented 5 years ago

Thanks for then comment @wangjin200 . When you set FPS greater than the original number, the video will become faster and shorter in length relative to the original. Also, the frame_detection_interval is not processed on per second basis. By design, the interval is calculated through the entire frames of the video input.