about the triplet loss layer

[GuitarYourself]

in your example,the triplet loss layer has two bottom, but it need 3 bottoms as input, can u give a complete example. thanks very much.

[DontStopMeNow]

Hi, i have a same problem. Also, how to give input data? Using ImageData layer without shuffle? Thank you in advance.

[tyandzx]

that's my bad, I should have mentioned it. Actually 2 bottoms are enough for triplet loss layer. But when preparing your train_val file list, not matter what kind of data layer you use(such as image_data_layer, data_layer), caffe will check to ensure that your train_val file list provides a image path and corresponding label. So there is alway a blob called "label". However, when we use sample_triplet_layer, we already make items in train_val file placed in label order, which means we can prepare triplet without label information. That make blob "label" un-used. If we don't provide a layer take blob "label" as bottom, annoying information will be printed during the whole training phase. That's why I made triplet_loss_layer having 3 bottoms: Just make blob "label" information disappear from the screen. @GuitarYourself

[GuitarYourself]

@tyandzx so, can i change 3 to 2 in ExactNumBottomBlobs() function?

[tyandzx]

there's still a more graceful way to solve it. You can make triplet_loss_layer have 2 bottoms. but use a "silence layer" provided by caffe and make blob "label" as its bottom.@GuitarYourself

[tyandzx]

Of course you can.@GuitarYourself

[tyandzx]

You're right. What I do is preparing a script to automatically generate train_val file for several epochs before training. In each epoch, re-group the combination of labels in each batch and shuffle the order of items in each label. That makes triplets be more variant and help to increase performance. During training, do not shuffle the list.@DontStopMeNow

[GuitarYourself]

@tyandzx i get it, thanks very much

[DontStopMeNow]

@tyandzx So, i can add label layer as bottom of tripletloss? Finally my input list ordered by label should looks like:

/home/moon/work/data/2874946_008.jpg 3382
/home/moon/work/data/2874946_016.jpg 3382
/home/moon/work/data/2874946_005.jpg 3382
/home/moon/work/data/2874946_004.jpg 3382
/home/moon/work/data/2874946_021.jpg 3382
/home/moon/work/data/2874946_014.jpg 3382
/home/moon/work/data/2874946_011.jpg 3382
/home/moon/work/data/2874946_001.jpg 3382
/home/moon/work/data/2874946_012.jpg 3382
/home/moon/work/data/2874946_025.jpg 3382
/home/moon/work/data/1890655_017.jpg 5988
/home/moon/work/data/1890655_005.jpg 5988
/home/moon/work/data/1890655_001.jpg 5988
/home/moon/work/data/1890655_004.jpg 5988
/home/moon/work/data/1890655_016.jpg 5988
/home/moon/work/data/1890655_011.jpg 5988

And .prototxt :

layer {
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  transform_param {
    mean_value: 127.5
    mean_value: 127.5
    mean_value: 127.5
    scale: 0.0078125
    mirror: true
  }
  image_data_param {
    new_height: 112
    new_width: 96 
    source: "new_train"
    batch_size: 60
    shuffle: false
  }
}
...
some conv
...
layer {
  name: "fc5"
  type: "InnerProduct"
  bottom: "res5_6"
  top: "fc5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 512
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "sample_triplet"
  type: "SampleTriplet"
  bottom: "fc5"
  top: "triplet"
  sample_triplet_param {
    label_num: 6
    sample_num: 10
  }
}
layer {
  name: "triplet_loss"
  type: "TripletLoss"
  bottom: "fc5"
  bottom: "triplet"
  bottom: "label"
  top: "triplet_loss"
  triplet_loss_param {
    margin: 0.1
  }
  loss_weight: 1
}

But losses increase in couple iterations and stop after that around 0.86 . Maybe I'm doing something wrong with triplet loss.

[tyandzx]

I think your list looks ok. I'm not quite sure about the problem. Maybe your can: 1) first build test case to checkout whether triplet_loss_layer can pass. 2) if 1) passes, use mini network such as LeNet and triplet loss layer to test a simple task and see whether it works. 3) smaller margin and lower learning rate may helps.@DontStopMeNow

[DontStopMeNow]

@tyandzx thank you so much. I will try your tips.

[GuitarYourself]

@tyandzx i have a question about the loss value, when i set margin for 0.5, the loss starts with about 0.48 and when i set margin for 1,the loss starts with 0.96, and in both these situations, the min loss is about 25% smaller than the started loss, and it seems very unstable, for example from 0.37 to 0.44 then to 0.39. Does this ok? or i have set the wrong param?

[tyandzx]

1)I don't think margin should be that huge case we use cosine metric(rather than l2 metric), the largest margin can only be 1. So you may need to lower the margin. 2) initial loss being smaller than margin is also possible(which depends on your task), it means intra-class samples locating more closer than inter-class samples but not that close to satisfy the margin constraint. @GuitarYourself

[GuitarYourself]

@tyandzx i reviewed your code in ComputeDiff_gpu, dose this function calculate the sin（which is the gradient of cos）,however, according to my calculate , this function seems calc the 1./x_1_norm*sin. Does that right?

[tyandzx]

Actually, ComputeDiff_gpu compute the diff of (A_T B) / (A_norm B_norm) w.r.t A @GuitarYourself

[Awadhesh42]

F0411 20:43:48.466460 4484 layer.hpp:349] Check failed: ExactNumBottomBlobs() == bottom.size() (2 vs. 3) SoftmaxWithLoss Layer takes 2 bottom blob(s) as input. Check failure stack trace: ./experiments/scripts/wpal_net.sh: line 39: 4484 Aborted (core dumped) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${NET}/solver.prototxt --weights ${WEIGHTS} --db ${DB} --setid ${DB_SET} --outputdir data/snapshots/${NET}/${DB_SET} --iters ${ITERS} ${EXTRA_ARGS}

HOW TO GIVE INPUSTS FROM BOTTOM??

[Yangel-hide]

What I do is preparing a script to automatically generate train_val file for several epochs before training. In each epoch, re-group the combination of labels in each batch and shuffle the order of items in each label.

你好，我想知道你是怎样在每个epoch开始之前重新生成train_val文件的？ caffe会在每个epoch开始之前重新读取一次train_val文件吗？脚本怎么判断当前epoch已经结束？