What is the use of mapping?

[CodeW1zard]

Hi, I am new to neural style transfer and confused about mapping in this paper. I've learned that mapping establishes an relationship between content layer and target layer. But I don't know how it could help the reconstruction.

So I turned to read the code.

        local match, correspondence = 
            cuda_utils.patchmatch_r(input, target, params.patchmatch_size, 1)
        match:cmul(mask) --match = elementwise multilication of match and mask
        local target_gram = gram:forward(match):clone()
        target_gram:div(mask:sum()) -- target_gram/mask:sum()

        local norm = params.normalize_gradients
        local loss_module = nn.StyleLoss(params.style_weight, target_gram, norm, mask_image):float():cuda()

In the reconstruction part of neural_gram.lua, target_gram is the formed by the patchmatch's result match. However, in neural_paint.lua, target_gram seems to come from BP, and BP is just the feature extracted by the pretrained VGG net.

    if i == #layers then -- i = 5, relu5_1
      print("  Initializing NNF in layer ", i, ":", name, " with patch ", params.patchmatch_size) 
      print("  Brute-force patch matching...")
      local init_corr = cuda_utils.patchmatch(N_A, N_BP, params.patchmatch_size) --[x1, y1, x2, y2,...,xn, yn]
      local guide = image.scale(style_image, w, h, 'bilinear'):float():cuda()
      print("  Refining NNF...")
      -- enforce spatial consistency
      corr = cuda_utils.refineNNF(N_A, N_BP, init_corr, guide, tmask, params.refine_size, params.refine_iter)
      mask = cuda_utils.Ring2(N_A, N_BP, corr, params.ring_radius, tmask)

      curr_corr = corr 
      curr_mask = mask       
    else -- i = 4, relu4_1
      print("  Upsampling NNF in layer ", i, ":", name)
      curr_corr = cuda_utils.upsample_corr(corr, h, w)
      curr_mask = image.scale(mask:double(), w, h, 'simple'):cudaInt()
    end   

    table.insert(match_features, BP)
    table.insert(match_masks, curr_mask)
  end 

Meanwhile, the whole patchmatch part has two results, i.e. curr_corr and curr_mask. But it seems that only 'curr_mask' is saved while the other one is abandoned. In my opinion, curr_mask is just the mask adjusted to the patchmatch and curr_corr is more important.

Could anyone kindly help me and give me some explanations? Thanks a lot.