Evaluation metrics

[abhijeet00]

Hi, Good Repo!

I have few question regarding metrics :

1. Can you please provide npy files which you used in metrics.ipynb?

2. Does metrics requires only predictions or both ground truth and predictions?

3. How many predicted images were used for each model for calculating metrics ?

[yashjain7856]

Hi Abhijeet!

The shape of the npy files used in the metrics was (1000,9,5). There are 1000 layouts for each model. Each layout has 9 boxes which are denoted by 5 parameters. The format is [c x y w h] where,

• c is the class of the box.
• x and y are the coordinates for the top left corner of the box.
• w and h are the width and height of the box respectively.

To calculate the metrics, only predictions are needed. However, the metrics should be close to the original data and that's why ground-truth metrics are also needed. In our analysis, we have normalized the metrics w.r.t. the ground truth.

1000 layouts from the original PublayNet Dataset and 1000 predictions from each model were used to calculate the metrics.