Add functionality to existing or new page that compares sizes of real paintings with photos

[ThijsVroegh]

Ratios branch

We want to be able to derive/estimate the absolute area of the figures in the real paintings, so that we then can compare the absolute areas between these paintings. The idea is that similar absolute areas indicate the usage of the same template used to reproduce a painting . We therefore have to add functionality to an existing or new page that does this. The approach is as follows:

- Lets denote 2 paintings as P1 and P2 and their corresponding photos as F1 and F2. We would like to verify whether the same template T has been used to produce the same "connected component" (in image processing lingo) shape S. That shape consisting of several touching figures (in the reseach lingo) appears is shape S1 in P1 and S2 in P2 respectively. - If the same T is used then the area of S1 should be the same as the area of S2 or the ratio of their areas should be 1 - However, we do not have access to P1 and P2 (hence cannot measure the areas of S1 and S2), but have access to F1 and F2 and we do have the dimensions of both painting and photos in cm. We also have the size of the photos in pixels and the dpi (which relates the sizes in cm and in pixels). Lets denote the shapes in the digital photos as SF1 and SF2. - We need to check how close is the ratio of corrected areas (calculated in number of pixels with the DPI and scaling correction) of SF1 and SF2 to 1! The areas in number of pixels for the shapes (connected components after the image segmentation) can be calculated by using any standard image processing library, e.g. using skimage: ``` import skimage as ski labeled_image = ski.measure.label(binary_mask) object_features = ski.measure.regionprops(labeled_image) ``` "area" is one of the object_features (region's properties) - The correction/normalization of the areas with the pixel size can de done by dividing the physical dimension to the pixel dimension. That is the dpi. - The scaling factor is a bit more tricky to calculate precisely, not knowing if the digital image has undergone some geometric post-processing such as cropping. (The real painting dimension and the photo ratios indicate some discrepancies, though not so big). One way is to use the scale factor from the homography derived from SIFT or other feature matching method. However, we are not matching exactly the same images hence, we cannot rely to a perfect precision. - Alternatively we can use the average between the ratios as estimation of the scaling factor. Then the corrected areas could be: B = A x (pixel size/ dpi) x scaling factor Where A = in amount of pixels

• [x] Extract dpi and size from images
• [x] Create function to calculate corrected area (def calculate_corrected_area(image, real_size_cm, photo_size_cm, dpi)
• [ ] Decide where to integrate this functionality; exisitng page or new page?
• [x] Add heatmap to depict results on similarity on painting area ratio
• [ ] After possible alignment, all images become equal in size. Does this have a negative effect on the corrected areas calculated from the aligned images? How to best cope with this? How to adjust the scaling factor calculation if images are resized? See the comments in the details below-> Ask Elena!

- The alignment function potentially resizes images to match the base image. This resizing can affect the pixel dimensions of the images, which in turn impacts the corrected area calculations. When images are aligned and resized, the areas in pixels might no longer directly correspond to the areas in the original paintings. - The calculation of the corrected area takes into account a scaling factor based on the real size of the painting and the size of the photo. However, after alignment, the scaling factor may need to be adjusted to account for any resizing that occurred during alignment. - If the alignment method alters the aspect ratio of the images, it could distort the shapes within the images, leading to inaccurate area calculations. The code should ensure that the aspect ratio of the images is preserved during alignment. - The calculation of corrected areas depends on the DPI of the images. If the DPI is not consistent across images or is altered during alignment, this could lead to incorrect area calculations. - The code uses the real size of the painting and the size of the photo to calculate the scaling factor. If the alignment process changes the photo size (in terms of pixels), this scaling factor needs to be recalculated based on the new dimensions. - The code allows the user to select different alignment methods. However, each method might affect the images differently. For example, feature-based alignment might involve more significant geometric transformations compared to Fourier Mellin Transform. The impact of the chosen method on the corrected area should be considered.

[ThijsVroegh]

@stefsmeets I also added the functionality as described above. What do you think would be the best place for this? In the alignment page? Or create a new page? I still have to speak to Elena about the consequence of alignment for this comparison before making a PR..

[stefsmeets]

That depends on how you want to implement it. I think we could add the size/dpi as metadata and use that in the aligment if available. We will have to keep any changes to the size in mind during the aligment and afterwards.

[ThijsVroegh]

Ok!