Questions about OFD strategy

[Carringbrinks]

Hello Thanks for contributing to MODNet project. I want to know if the code about OFD strategy is in the inference file? How to apply real-time inference about OFD strategy?

[ZHKKKe]

Hi, thanks for your attention. The code of OFD is not included now, but it will coming soon. :)

[xafha]

Hi, thanks for your attention. The code of OFD is not included now, but it will coming soon. :)

请加速啊，谢谢。

[luoww1992]

+1

[Carringbrinks]

Hello , I am sorry to reply to you now. I have some questions about your ModNet training data set. I want to consult you.  I made 200k data sets by myself. The results of training with your code are still not as effective as in your paper.  My data set is composed of foreground and background. I have a total of 2000 foregrounds and 100 backgrounds. Can you tell me how your data set is made? I'm looking forward to your reply, thank you!

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年2月20日(星期六) 下午2:46 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

Hi, thanks for your attention. The code of OFD is not included now, but it will coming soon. :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks The first step is enlarge your background set... We use more than 500k backgrounds for composition (Place-365 dataset).

[Carringbrinks]

Hello, thank you for your reply You said on github that you used 100k data sets. how many did you use for the foreground and background?

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月1日(星期四) 下午4:52 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks The first step is enlarge your background set... We use more than 500k backgrounds for composition (Place-365 dataset).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks We use only about 3k labeled foreground. We composite 100k training samples. For each sample, we use a background randomly selected from 500k backgrounds.

[JerryDeepl]

@Carringbrinks The first step is enlarge your background set... We use more than 500k backgrounds for composition (Place-365 dataset).

Hello, could you share how to pick up the background images from Place-365 dataset ? thanks

[ZHKKKe]

@JerryDeepl We use a face detection model to remove all images included persons.

[JerryDeepl]

@ZHKKKe Got it. Thanks

[Carringbrinks]

Hello  I am currently writing my graduation thesis on matting. I would like to ask whether the comparison algorithm in your MODNET paper is reproduced by yourself or is the code requested from the original author?

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月14日(星期三) 下午2:40 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@JerryDeepl We use a face detection model to remove all images included persons.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks For the methods without official code, we reproduced them.

[Carringbrinks]

Hello,

I’m currently doing my master’s thesis.

But for these algorithms that are not open source, even if I reproduce them, I don’t have so many GPUs to train these models

So do you give me a copy of the pre-trained model of the SHM algorithm and LFM algorithm ?

I need it very much.

If you can give it to me, I am very grateful to you. If not, it doesn't matter.

Thank you

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月19日(星期一) 晚上8:58 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks For the methods without official code, we reproduced them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks I am sorry that I do not have permission to share the pre-trained models as these models are trained on the private dataset. (These models are not constrained by any license, so others may use them for any purposes).

[Carringbrinks]

Okay, it doesn't matter.

Did you ask the original author for the model structure?

Or you reproduced the model structure and then trained it yourself?

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月22日(星期四) 中午1:15 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks I am sorry that I do not have permission to share the pre-trained models as these models are trained on the private dataset. (These models are not constrained by any license, so others may use them for any purposes).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks We reproduced the model structures and trained them ourselves.

[Carringbrinks]

Ok thanks for your reply

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月22日(星期四) 下午2:01 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks We reproduced the model structures and trained them ourselves.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Carringbrinks]

Hello I now use 3000 foregrounds and 50000 backgrounds to randomly compose a training set of 160,000 The model is over-fitted, and the real-time stream generalization effect is very poor I want to use your soc strategy to fine-tune, but the loss value soon becomes nan  How should this be solved? Hope you can solve it, Thank you

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月22日(星期四) 中午1:15 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks I am sorry that I do not have permission to share the pre-trained models as these models are trained on the private dataset. (These models are not constrained by any license, so others may use them for any purposes).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks Based on my experience, there are several problems may cause NaN loss in SOC:

1. The loss is too large, please reduce the soc loss scale.
2. The target domain is too different from the source domain (If you can share some images of the two domains, I may give you some judgement).
3. There are some bad samples in the unlabeled dataset, e.g., there are no portrait in the input image.

[Carringbrinks]

My training set is like the format in the train_set folder, and my soc strategy training set is like the format in the soc_train_set folder. For the scale value of the loss function, I set all of them to 1 before, and it became nan after training. There is no cartoon data set in my training set, but there is in my soc strategy. Will this be the problem?

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月29日(星期四) 下午4:45 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks Based on my experience, there are several problems may cause NaN loss in SOC:

The loss is too large, please reduce the soc loss scale.

The target domain is too different from the source domain (If you can share some images of the two domains, I may give you some judgement).

There are some bad samples in the unlabeled dataset, e.g., there are no portrait in the input image.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks Yes... It may will... The domain of the cartoon data is to far from the real-world data.

[Carringbrinks]

Hello Bother you again I want to ask you another question about the modnet data set. 

I want to train a portrait matting model. I have prepared 4,500 person foregrounds, 2,000 backgrounds, and randomly synthesized 90,000 pictures. But the trained model It has been over-fitting, and the effect of real-time video streaming is particularly poor.  What is wrong with my data set?

My background is also the place-365 dataset,  But when I only use 100 backgrounds on the Internet, the generalization effect of the model's real-time video stream is better.  Do you know where the problem is?

In addition, can you tell me what kind of prospects your data set has?

Looking forward to your answer, thank you

------------------ 原始邮件 ------------------ 发件人: "ZHKKKe/MODNet" @.>; 发送时间: 2021年4月30日(星期五) 凌晨1:22 @.>; @.**@.>; 主题: Re: [ZHKKKe/MODNet] Questions about OFD strategy (#68)

@Carringbrinks Yes... It may will... The domain of the cartoon data is to far from the real-world data.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ZHKKKe]

@Carringbrinks Hi, for your questions: Q1: What is wrong with my data set? There is also an over-fitting problem in our model, which is why we need a SOC strategy. Besides, have you calculated the foreground color before image composition? This step is vital for getting more realistic training samples. If the foreground color is not be calculated, the gap between the synthetic samples and the natural images will be larger.

Q2: Do you know where the problem is? What's your input size? The resolution of the image in OpenImage is usually small. Therefore, you must upsample them for image synthesis, which will result in blurred backgrounds. You can try to replace the background set with some open source image super-resolution data set.