search

jingtingEmmaLi / CAS-ME-3

30 stars 2 forks source link

Onsets, apexes and offsets #4

Open tvaranka opened 2 years ago

tvaranka commented 2 years ago

I have a few questions regarding the onsets, apexes and offsets in part_A ME.

  1. There are several samples (55 to be precise), where the onset and the apex are the same. How is this possible?

  2. The offsets seem to be very long for most of the samples. There is one sample (Table below, row 1) that has a length of 24 (730 / 30) seconds and multiple samples (a total of 72) that are over 2 seconds long. How can these be justified as MEs? Is the criteria of (apex - onset < 250ms) too loose for cases like these?

(Moreover, there seems to be a large bias where the apex is always significantly closer to the onset than the offset.)

Here are a few samples with very long durations. (n_frames = number of frames) image

jingtingEmmaLi commented 2 years ago

Dear Tvaranka,

Regarding the first issue, this is because the first few frames of the video are not recorded (as mentioned in the previous issue, the first 7 frames or so of the video may be lost). The frame that the coder sees is already the maximum magnitude of that expression.

Regarding the second issue, in our coding experience, micro-expressions are usually distinctly characterized during the onset-apex phase, peaking very quickly. But the change of apex-offset phase is very diverse, some may revert to neutral face quickly and some may change slowly. Therefore, we set a criterion of apex-onset <250ms for micro-expression classification.

In the task of intelligent analysis of micro-expressions, if this part (offset is too far from apex) affects the analysis, onset-apex-(apex+(apex-onset)) can be used as the interval for analysis.

Best wishes, Jingting

tvaranka commented 2 years ago

Makes sense. There are however 4 cases where the onset and apex are the same with the frame 1. Do these cases have dropped frames also but the start has been set to frame 1? As for most cases the start is around the 7th or 8th frame. image

Thanks a lot!

jingtingEmmaLi commented 2 years ago

It is the similar situation. In fact, the 7th or the 1st frame is the same situation for the coder. The time/the frame they find the expression is already the most significant of the expression. For the case where the expression was present in the first frame, it could be that the subject was presenting a facial expression before the video recording started. And this expression was captured at the moment when the video recording started.

tvaranka commented 2 years ago

"For the case where the expression was present in the first frame, it could be that the subject was presenting a facial expression before the video recording started." Could it be possible that this was a macro-expression and not a micro-expression? Since the segment just started it is not possible to know which one it could be?

jingtingEmmaLi commented 2 years ago

Yes, it could be. During our coding process, because the intensity of these four expression movements in this particular case was also low, our coder agreed to label them as micro-expressions. If you find it bothersome, you could exclude it from the data analysis.

tvaranka commented 2 years ago

Thanks a lot for the answers! Very happy to see you have also though about the raised concerns during the coding process.

I will be experimenting and seeing whether excluding some of the discussed samples affect the results and in what ways.