search

stringtron / quantative_style

Improving Style Transfer With Calibrated Metrics
https://arxiv.org/abs/1910.09447
MIT License
16 stars 5 forks source link

Can you use this metric for images with higher resultion than 481x321? #3

Open JonWofr opened 3 years ago

JonWofr commented 3 years ago

Hey,

first of all thank you for your amazing work.

I was wondering if you could also use images with a higher resolution. My understanding right now is that in order to get the C statistic you have to stylize images from the BSDS500 test set which are of low resolution (481x321). The generated images should also have the same resolution (i.e. 481x321) because otherwise one could not apply the contour detection algorithm and would eventually not be able to calculate the C statistic. So it is not possible to calculate the C statistic with images which have a higher resolution, right? Or am I missing something?

shuait commented 3 years ago

Hi,

Thanks for your interest in our work. I will try to address your questions below.

  1. "The generated images should also have the same resolution."

The generated images have the same dimensions as the content images. We set this because that the ground-truth content contour maps are for that dimensions. I think if you can scale the countour annotation maps to the desired dimensions, you can still apply our method to a higher image resolution.

  1. "So it is not possible to calculate the C statistic with images which have a higher resolution"

Calculating the base C statistics requires you have an image with its human-annotated contour maps. As from my previous answer, you may simulate a contour map by scaling it. However, we haven't conducted experiments to check if human contour annotations on a scaled (i,e, 481x321 -> 321x481) image are the same as the scaled contour maps.

Hope I have answered your questions. Let me know if you need more help.

JonWofr commented 3 years ago

Okay, understood. Thank you so much for your response.

JonWofr commented 3 years ago

@shuait Another question regarding the same topic came to my mind.

In your paper you state that you resized the smaller side of the content- and style-images to a resolution of 512px before applying any NST-method. Additionally in the code for the E-statistic all images are resized to 512px before PCA is done. Therefore I assume that the outputs of the NST-methods are the same size as the inputs (i.e. smaller side = 512px). But in order to be able to apply the C-statistic the output of a NST-method has to be the same size as the human-annotated contour maps (as clarified in your previous answer).

So do you resize the output of each NST-method to the required resolution of 481x321 or 321x481 before applying the C-statistic? And what are your motivations behind the resizing in the first place? I don't understand why you do not simply use the initial resolution of 481x321 or 321x481 and use them together with accordingly scaled style-images for the NST-methods. By that you would not have to upscale before NST and downscale the result to be able to use the C-statistic afterwards. And I don't understand why you upscale the images for the E-statistic. You could also compute PCs, make projections and calculate max-F on the original resolution.

Thank you in advance for any thoughts on this :)