Genetic Algorithm

[genbing99]

Hi, I've read your BDCNN paper. However, I couldn't find the code for the genetic algorithm that you used in the paper. Can you share the part where GA is implemented?

[tvaranka]

I found the code for the genetic algorithm from https://github.com/MLDMXM2017/GA_ME. I tried running the code but wasn't able to reproduce the results shown in the paper. Emailed the authors, but they weren't able to help with the issue either. Not exactly sure where the issue is but have an idea. If you continue with the work please let me know!

[genbing99]

Yes, I've checked with the authors and they only have Matlab code. Would be glad if you are able to replicate the code of the Genetic Algorithm using Python and share it with me.

[tvaranka]

Let me do some tests and see if I can get it to work. I will port it to python if it works as well.

Have you attempted to reproduce the results of BDCNN without the genetic algorithm?

[genbing99]

No, I'm only interested in the genetic algorithm since it is able to improve the performance so much

[tvaranka]

I recreated my experiments and found that I had a bug (the ordering of video clips changed when I was sorting subjects with pandas, fixed by sorting by [subject, index]), but was able to fix it now.

I was now able to recreate the the method to a reasonably close F1 macro of 0.77. There is however a massive but.

Experiment The method uses features from STSTNet but does not describe how the features were extracted. The co-authors were also unable to tell me how they were extracted when asked over email.

There are two ways of doing this. There is the wrong way and the right way.

1. The right way. Train STSTNet using LOSO and extract the features one by one for each subject.
2. The wrong way. Train STSTNet with most (if not all) subjects and extract the features for all at once.

After experimenting with both feature extraction techniques the results are as follows. For 2 the result is 0.7740 F1 macro, while for the 1 the result is 0.4169 F1 macro.

From these results I would believe that the authors used method 2 to get their results and have a data leakage issue.

[genbing99]

Thanks for the interesting findings! Hope the authors can clarify this issue.

[tvaranka]

Sent an email to the authors regarding both the genetic algorithm as well as the BDCNN as I also had trouble with that (got an F1-score of about 0.71). Let's hope they can help!

[tvaranka]

Have not gotten a reply since I sent the email to the authors. @genbing99 I have replicated the genetic algorithm and it is now in Python. Let me know if you still need it

[genbing99]

Yes, I'm interested in it. Can you share the code through email (genbing67@gmail.com) with me?

[ZhouQuan-one]

I recreated my experiments and found that I had a bug (the ordering of video clips changed when I was sorting subjects with pandas, fixed by sorting by [subject, index]), but was able to fix it now.

I was now able to recreate the the method to a reasonably close F1 macro of 0.77. There is however a massive but.

Experiment The method uses features from STSTNet but does not describe how the features were extracted. The co-authors were also unable to tell me how they were extracted when asked over email.

There are two ways of doing this. There is the wrong way and the right way.

1. The right way. Train STSTNet using LOSO and extract the features one by one for each subject.
2. The wrong way. Train STSTNet with most (if not all) subjects and extract the features for all at once.

After experimenting with both feature extraction techniques the results are as follows. For 2 the result is 0.7740 F1 macro, while for the 1 the result is 0.4169 F1 macro.

From these results I would believe that the authors used method 2 to get their results and have a data leakage issue.

Hi, I've been replicating the BDCNN experiment recently and have encountered a similar problem to yours. Regarding the four optical flow features mentioned in the article, the first three from STSTNET and vis from FLOWNET, do you know how they work? Do you have a relevant program to get them?

[tvaranka]

In their paper they use a quadruple (u, v, os, vis), where vis is from FLOWNET, but where u, v and os come from is not certain. I glanced over the paper and couldn't see them mentioning which optical flow was used. STSTNet is for classifying micro-expressions using optical as the input, so the optical flow is not from there.

Based on the figures, I would say the optical flow is from [1]. That is also the method I did my experiments on. The matlab code for [1] is publicly available and I have used Matlab engine to call the code directly from Python. Let me know if you are interested and I can share the code with you.

[1] Deqing Sun, Stefan Roth, and Michael J. Black. 2010. Secrets of optical flow estimation and their principles. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2432–2439. https://doi.org/10.1109/CVPR.2010.5539939

[ZhouQuan-one]

In their paper they use a quadruple (u, v, os, vis), where vis is from FLOWNET, but where u, v and os come from is not certain. I glanced over the paper and couldn't see them mentioning which optical flow was used. STSTNet is for classifying micro-expressions using optical as the input, so the optical flow is not from there.

Based on the figures, I would say the optical flow is from [1]. That is also the method I did my experiments on. The matlab code for [1] is publicly available and I have used Matlab engine to call the code directly from Python. Let me know if you are interested and I can share the code with you.

[1] Deqing Sun, Stefan Roth, and Michael J. Black. 2010. Secrets of optical flow estimation and their principles. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2432–2439. https://doi.org/10.1109/CVPR.2010.5539939

I am glad to receive your reply, I am interested in them and would be happy if you would share them with me via email（583540165@qq.com）

[LSC00]

@tvaranka Hello, can you share the four optical flow features and the genetic algorithm in python? Hope to receive your reply, thank you!

[tvaranka]

Hey, I have made the genetic algorithm publicly available here. The code uses a different protocol, but it can be modified to use the MEGC protocol similar to the paper.

The optical flow features can be computed with the extract_optical_flow.py script in this folder. It can be a bit tricky, so I can also share the OF features directly if you share your email.

[LSC00]

I am glad to receive your reply, I am interested in them and would be happy if you would share them with me via @.***）  

漠然走过 却留心看你风景 @.***

 

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2022年12月27日(星期二) 中午1:52 收件人: @.>; 抄送: "漠然走过 @.>; @.>; 主题: Re: [MLDMXM2017/BDCNN] Genetic Algorithm (Issue #1)

Hey, I have made the genetic algorithm publicly available here. The code uses a different protocol, but it can be modified to use the MEGC protocol similar to the paper.

The optical flow features can be computed with the extract_optical_flow.py script in this folder. It can be a bit tricky, so I can also share the OF features directly if you share your email.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[LSC00]

@tvaranka I am glad to receive your reply, I am interested in them and would be happy if you would share them with me via 342975250@qq.com

[cherish6dream]

@tvaranka Hi, could you share the four optical flow features with me via 2968589843@qq.com? Hope to receive your reply, thank you!

[ylf012]

@tvaranka Hi, could you share the four optical flow features with me via 2300768037@qq.com? Hope to receive your reply, thank you!

[emm110112]

@tvaranka Hi, thank you for your kind answer, I am also very interested in the work of this paper, can you share the four optical flow characteristics with me through the 1710622872.@qq.com? Hope to hear from you, thank you.

[SQQ531]

@tvaranka Hi, could you share the four optical flow features with me via 1401127222@qq.com? Hope to receive your reply, thank you so much!

[liuxiuqi98]

@tvaranka Hi, could you share the four optical flow features with me via 1396556220@qq.com? Hope to receive your reply, thank you so much!

[ZAMI900384]

嘿，我已经在这里公开了遗传算法。该代码使用不同的协议，但可以修改为使用类似于论文的 MEGC 协议。

可以使用此文件夹中的 extract_optical_flow.py 脚本计算光流特征。这可能有点棘手，所以如果你分享你的电子邮件，我也可以直接分享OF功能。

@tvaranka Hi, could you share the four optical flow features with me via songfan299@gmail.com? Hope to receive your reply, thank you so much!

[xc0719]

@tvaranka Hi, could you share the four optical flow features with me via 2979228778@qq.com? Hope to receive your reply, And I have some other questions that I wonder if you have time to answer.thank you so much!

[w1139571193]

@tvaranka Hi, thank you for your kind answer, I am also very interested in the work of this paper, can you share the four optical flow characteristics with me through the 1139571193.@qq.com? Hope to hear from you, thank you.

[w1139571193]

@liuxiuqi98 Hi, thank you for your kind answer, I am also very interested in the work of this paper, can you share the four optical flow characteristics with me through the 1139571193.@qq.com? Hope to hear from you, thank you.

[MERxX233]

您好，您能通过 （1815036039@qq.com） 与我分享四个光流特征吗？希望收到您的回复，谢谢！