search

tyiannak / multimodal_movie_analysis

A Python Library for Multimodal Analysis of Movies and Content-based Movie Recommendation
25 stars 8 forks source link

Add clustering to wrapper #47 #48

Closed apoman38 closed 3 years ago

apoman38 commented 3 years ago

What this PR does: Add a new function named clustering. The clustering function takes the average vector for each folder of video_class_predict_folder() and applies clustering. After the process is finished, save the clustering algorithm's predictions and save a png file with clusters' plot.

How to QA: I prepared a sanity test of the above process. Below are the steps for sanity's test implementation:

  1. Train a binary classification problem (Static, Non-static) with train.py or download a pre-trained model from the link below: https://drive.google.com/drive/folders/1fdK4F1_Nd1aDRCxKS98aKdvit1OgOx6f?usp=sharing

  2. Download the videos: youtube-dl -o non_static1 'https://www.youtube.com/watch?v=668nUCeBHyY' youtube-dl -o non_static2 'https://www.youtube.com/watch?v=if3VeU9i8hE&t=19s' youtube-dl -o non_static3 'https://www.youtube.com/watch?v=2-YN-hRUtfI' youtube-dl -o static1 'https://www.youtube.com/watch?v=gQqpvZMtUTY' youtube-dl -o static2 'https://www.youtube.com/watch?v=CU4flrPwFCI' youtube-dl -o static3 'https://www.youtube.com/watch?v=p7QCPgdngBo'

  3. Create directories to store all videos: mkdir non_static1 non_static2 non_static3 static1 static2 static3

  4. Refresh the folder (important step)

  5. Move videos to directories: mv non_static1.mp4 non_static1 mv non_static2.mkv non_static2 mv non_static3.mkv non_static3 mv static1.mp4 static1 mv static2.mp4 static2 mv static3.mkv static3

  6. Shot generation: python3 shot_generator.py -f non_static1/non_static1.mp4 python3 shot_generator.py -f non_static2/non_static2.mkv python3 shot_generator.py -f non_static3/non_static3.mkv python3 shot_generator.py -f static1/static1.mp4 python3 shot_generator.py -f static2/static2.mp4 python3 shot_generator.py -f static3/static3.mkv

  7. Some videos files from the shot generator doesn't work, so delete them:

  1. Run wrapper.py: python3 wrapper.py -i non_static1 non_static2 non_static3 static1 static2 static3 -m Algorithm -o output.csv

How to QA:(some extra check) Check all the scenarios of wrapper's inputs e.g.

tyiannak commented 3 years ago

"Some videos files from the shot generator doesn't work, so delete them" This needs to be checked. Why?

apoman38 commented 3 years ago

You are right. I added a check if there is a corrupted video to skip it. But and in this case, the original files must be deleted. Some videos have a single shot, so delete only the original file from videos with many shots. Replace the commands from step 7 with the commands below:

  1. rm non_static1/non_static1.mp4 rm non_static2/non_static2.mkv rm non_static3/non_static3.mkv