To analyze a movie in terms of its auditory content, do the following:
cd analyze_audio
python3 analyze_audio.py -f movie.wav
Note: You will need to create a folder in analyze_audio/segment_models
where
you will store your audio SVM segment classifiers. See analyze_audio/readme.md
for instructions on how to train these audio classifiers. Currently the audio analysis module
expects 5 audio classifiers: (1) a generic audio classifier (4-classes) (2) two speech emotion classifiers
and (3) two musical emotion classifiers.
To extract hand-crafted audio features run the following:
python3 analyze_visual.py -f ../V236_915000__0.mp4
The features are saved in npy files. The main functionality is implemented in function process_video
that extracts features from specific file.
See analyze_visual/Readme.md
for more details.
You can also train a supervised model of video shots (e.g. types of shots):
python3 train.py -v data/class1 data/class2 -a SVM
The following files will be saved to disk:
shot_classifier_SVM.pkl
the classifiershot_classifier_SVM_scaler.pkl
the scalershot_classifier_SVM_results.json
the cross validation resultsshot_classifier_conf_mat_SVC().jpg
the confusion matrix of the cross validation procedureAs soon as the supervised model is trained you can classify an unknown shot (or shots organized in folders):
python3 wrapper.py -m SVM -i test.mp4
The following script detects the change of the shots in a video file and it stores the respective shots in individual files.
It can be used in combination with the wrapper.py
script above to analyze a movie per shot.
python3 shot_generator.py -f data/file.mp4