search

CoinCheung / MFM

code for paper "Masked Frequency Modeling for Self-Supervised Visual Pre-Training" (https://arxiv.org/pdf/2206.07706.pdf)
MIT License
23 stars 2 forks source link

On the focal frequency loss. #3

Open shiliang26 opened 1 year ago

shiliang26 commented 1 year ago

Hi,

I noticed that you removed the weight that balances the losses of different positions, which is a part of the focal frequency loss.

I wonder why that is. Did you observe a performance degradation?

I'm working on a similar project that involves spectrum reconstruction, and could use your advice. Intuitively, given the unbalanced distribution of the spectrum, the weight seems a good choice.

Thank you!

CoinCheung commented 1 year ago

Yes, from my observation, the results achieved with weight is not so good as that without weight. But it only limits to my experiments, I feel it should depend on specific using case. I encourage you to try both, and use the better in your case.