search

rsanchezgarc / cesped

Cryo-EM Supervised Pose Estimation Dataset utilities
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Questions about evaluation and per-image normalization #2

Closed RuyiLian closed 5 months ago

RuyiLian commented 5 months ago

Hi, Thanks for developing such awesome benchmark! I am mainly working on natural images so I have some confusions of the details.

  1. In the evaluation, I do not understand why one half-set reconstruction is symmetry-aware while the other one is not: https://github.com/rsanchezgarc/cesped/blob/9d09bd095b053119ae175b5020de19d41c6eb8e0/cesped/evaluateEntry.py#L157-L161

  2. For the per-image normalization, I find it very different from the common practice of natural images. As demonstrated in your paper, the per-image normalization follows the standard Cryo-EM procedure. Could you provide a specific example of the standard procedure?

Thanks!

rsanchezgarc commented 5 months ago

Hi!

Thank you very much for your interest in our benchmark!

  1. This is a bug. Thank you for spotting it! Since it affects both the ground truth and the inferred poses in the same manner, it shouldn't have a huge impact on the numbers reported in the pre-print, but I will re-run the evaluation code and update the tables.
  2. This is what the Relion manual says for image normalization: "Particles will be normalized to a mean value of zero and a standard deviation of one for all pixels in the background area. The background area is defined as all pixels outside a circle with this given diameter in pixels (before rescaling). When specifying a negative value, a default value of 75% of the Particle box size will be used". You can read about the reasons here. My implementation for such a normalization is here: https://github.com/rsanchezgarc/cesped/blob/9d09bd095b053119ae175b5020de19d41c6eb8e0/cesped/particlesDataset.py#L270-L283
RuyiLian commented 5 months ago

Great! Thank you very much for your quick and detailed reply.