Auto-segmentation Current Shortcoming

[flueko]

I am now in the process of analyzing a larger group of more recent films by the use of auto-segmentation and I identified one pattern that indicates a current shortcoming.

Segments are sometimes not divided by color and in this case also result in extremely long ones that exceed the average segment length. 3105_1_1_SpringBreakers_2012_DVD_VIAN.eext.zip

[flueko]

By contrast 319 "Il gattopardo" worked extremely well.

[ghalter]

We have to think about other segmentation methods, such as shot detection, audio segmentation etc.

[ghalter]

We should think about more auto segmentations in the future, for the milestone I move this issue out.

[flueko]

There are still these shortcomings in auto segmentation:

• Some segments are too long
• Some segments are extremely short, sometimes just a black fade
• higher sensitivity for color change in dark scenes leads to too many segments in those scenes

[flueko]

For instance the logos at the start of the film are almost always segmented in many small parts.

[flueko]

And here one of these super short segments, they are easily overlooked if no screenshot is created of these short segments.

[ghalter]

Thanks for the well descriped issues :-) Ok. some approaches come to my mind:

Some segments are extremely short, sometimes just a black fade

maybe we can simply add a threshold, such that sequences are not shorter than some threshold which the user can set in the dialog (or during the creation of the segments)

higher sensitivity for color change in dark scenes leads to too many segments in those scenes

Currently, the clustering is performed on all three channels (L, C, H), maybe it helps if we could drop the L channel during the agglomeration?

Some segments are too long On what premise? The clustering takes only color into account, the longest segment in your screenshot above seems quite consistent when it comes to color.

[flueko]

For instance fades to black are usually an indication for a scene transition. But now they are often segmented into a very short segment.

[flueko]

maybe we can simply add a threshold, such that sequences are not shorter than some threshold which the user can set in the dialog (or during the creation of the segments)

I would rather set universal thresholds for smallest and longest segments and give users a possibility to override in case of specific films.

[flueko]

On what premise? The clustering takes only color into account, the longest segment in your screenshot above seems quite consistent when it comes to color.

For instance this scene is too long, and I can clearly spot some color changes from cold to warm that are related to change of location.

[flueko]

It relates to the segmentation being more sensitive in darker parts than in brighter parts.

[flueko]

Takes extremely long in new VIAN version, more than 20 minutes. Used to be much faster, maybe 4 to 5 minutes.