tyiannak / multimodal_movie_analysis

A Python Library for Multimodal Analysis of Movies and Content-based Movie Recommendation
25 stars 8 forks source link

Shot Evaluation: Create script that compares the 3rd argument of process_video() with the ground truth. #13

Closed tyiannak closed 3 years ago

tyiannak commented 3 years ago

@apoman38 Branch is here https://github.com/tyiannak/multimodal_movie_analysis/tree/%2313-Shot-Evaluation-Script

Algorithm to be implement Input:

Algo: gt = read_ground_truth_fromfile() , _, predicted_t = process_video(...)

time_tolerance = 0.5 for each timestamp g in gt: if g's minimum distance from predicted_t is < time_tolerance: count_gt_correct += 1

for each timestamp p in predicted_t: if p's minimum distance from gt is < time_tolerance: count_p_correct += 1

recall = count_gt_correct / total_timestamps_in_gt precision = count_p_correct / total_timestamps_predicted

Also, it should do the same if a folder is given (and count the AVERAGE among all videos + gt files)

apoman38 commented 3 years ago

Created a new file named "shot_evaluation.py" and a new folder that contains the timestamps of shots. The execution of the file is as follows:

The shot timestamps files must have the same name with the video file e.g if the the video file is "movie.mp4" the text file must be "movie.txt". The final output of script is the precision and recall and in case that many videos given by the user the output is the average of precision and recall of all files.

tyiannak commented 3 years ago

calc() function is wrong.

the loop as it is now (for timestamp_1,timestamp_2 in zip(annotated_shots,shot_change_times)) just goes through every element in one list and the respective element in the 2nd list.

tyiannak commented 3 years ago

I've debugged this. @apoman38 can u please extract recall and precision (manually) for tolerance = 0.5 for these results???

[0.0, 3.4034, 5.4054, 7.4074, 9.809800000000001, 11.611600000000001, 15.6156, 18.6186, 19.8198, 21.421400000000002, 22.8228, 25.625600000000002, 26.826800000000002, 28.028000000000002, 29.4294, 30.6306, 32.032000000000004, 34.200833333333335, 35.402033333333335, 41.80843333333333] Timestamps for actual shots: [0.0, 3.151, 5.15, 7.401, 9.65, 11.65, 13.401, 15.396, 17.646, 19.645, 21.147, 22.895, 24.896, 26.896, 28.896, 30.645, 32.145, 34.145, 36.396, 38.57, 41.865]

tyiannak commented 3 years ago

also @apoman38 pls never never never :-) use tabs.

apoman38 commented 3 years ago

Sorry for the appearance of the code. Next time I will deliver it in better form.

tyiannak commented 3 years ago

just kidding 👍

tyiannak commented 3 years ago

python3 shot_evaluation.py -f temp/f_f_small1.webm

@lobracost run this to see the error

tyiannak commented 3 years ago

merging and closing