search

mvondracek / VUT-FIT-POVa-2018-Pedestrian-Tracking

Computer vision system for tracking pedestrians in a scene observed by multiple cameras.
10 stars 4 forks source link

Experiments and evaluation for the final presentation in January #7

Closed mvondracek closed 5 years ago

mvondracek commented 5 years ago

(...) experiments and evaluation results. It is not enough to provide a solution to a task - you should evaluate how well it works. The evaluation can be quantitative (e.g. segmentation accuracy on a standard dataset) or qualitative (e.g. user feedback). Project report

related: #2

mvondracek commented 5 years ago

Evaluation

Following text summarises testing of most important parts of the implemented solution. We have analysed speed and precision of individual components. We have captured and used total of 22 own images from 3D scenes with walking people and 4 own videos from these scenes. Files are identified by scene number (prefix s1, s2, and s3), camera identification and position (f and front, m and side), number of people (single, multi). Therefore, our testing data consist of following files which are further described in testing/_data/README.md. 

testing_data/s1_front_d150_h50.jpg
testing_data/s1_front_d400.jpg
testing_data/s1_side_d500.jpg
testing_data/s2_f_x0y300.png
testing_data/s2_f_x0y600.png
testing_data/s2_m_x0y300.png
testing_data/s2_m_x0y600.png
testing_data/s3_f_side_multi_y600.png
testing_data/s3_f_side_single_F_y600.png
testing_data/s3_f_side_single_x0y300.png
testing_data/s3_f_side_single_x50y600.png
testing_data/s3_m_front_multi_bg.png
testing_data/s3_m_front_multi_y600.png
testing_data/s3_m_front_single_F_y600.png
testing_data/s3_m_front_single_x0y300.png
testing_data/s3_m_front_single_x50y600.png
testing_data/s3_single_30_front.png
testing_data/s3_single_30_side.png
testing_data/s3_single_49_front.png
testing_data/s3_single_49_side.png
testing_data/s3_single_51_front.png
testing_data/s3_single_51_side.png
testing_data/s3_f_side_multi.mov
testing_data/s3_f_side_single.mov
testing_data/s3_m_front_multi.mov
testing_data/s3_m_front_single.mov

People Detection in 2D -- OpenPose

Implementation of people detector based on OpenPose network was evaluated both for speed and precision. Detector is able to identify key parts of human body with precision of 10 pixels in testing images, these tests are implemented in test_povaPose.py unit tests. On the other hand, detection process was not very fast. \autoref{fig:openpose_speed} shows speed evaluation for different computers.

obrazek Figure 5: Speed evaluation for repeated detection of person in testing image.

Triangulation

Implementation of CameraDistanceTriangulation was tested and evaluated using unit tests located in test_triangulation.py. Achieved precision of determining location was within 10 cm delta, in some cases 15 cm delta, and in worst case within 35 cm delta in scene where people were walking in 6 m space.

mvondracek commented 5 years ago

@xstast24, @flaxh We will need further testing and experiments for evaluation during our presentation in January. Please prepare materials for the parts of our project that you are responsible for.

mvondracek commented 5 years ago

Presenatation date: 2019-01-09 Mo 15:00-17:00 @xstast24, @flaxh, please don't forget about it, Related: #1