The best results by using Resnet152

[yuanli2333]

Hi, according to the paper, all results in the paper are obtained by AlexNet. So what's the result on the three datasets if use Resnet152 as feature extraction model? I run your pytorch code by using Resnet152 as the base CNN model, but get worse results than your paper. Thank you very much if you can answer this question!!!

[caozhangjie]

You need to tune the parameters such as learning rate and the trade off for other base networks. You can also try to fix some layers of ResNet to prevent overfitting.

[yuanli2333]

You need to tune the parameters such as learning rate and the trade off for other base networks. You can also try to fix some layers of ResNet to prevent overfitting.

Yes, I fine-tuned on the ResNet, but didn't get a better result, but I also want to know the results you got, because I want to compare with your results.

[caozhangjie]

I mean try some hyper parameters such as learning rate and the trade off. And since ResNet os much more deeper than AlexNet, I suggest fixing the ResNet and only train the encoding layer.

[yuanli2333]

Thank you! zhangjie.

[soon-will]

@yl2488 How did you train the model? I fine-tuned on the ResNet50, the mAP is 0.7780. This is my experiment details: Model: ResNet50(the activation function is tanh) Loss: pairwise cross-entropy(without the weight to account for the data imbalance, with alpha==1) R: 5000(dataset is nuswide_81) epoch: 50(if go on training, the loss will be lower)

loss:

def pairwise_loss(outputs1,outputs2,label1,label2):
    similarity = Variable(torch.mm(label1.data.float(), label2.data.float().t()) > 0).float()
    dot_product = torch.mm(outputs1, outputs2.t())
    #exp_product = torch.exp(dot_product)

    mask_positive = similarity.data > 0
    mask_negative = similarity.data <= 0

    exp_loss = torch.sum(torch.log(1+torch.exp(-torch.abs(dot_product))) + torch.max(dot_product, Variable(torch.FloatTensor([0.]).cuda()))-similarity * dot_product)

    #exp_loss = torch.sum(torch.log(1 + exp_product) - similarity * dot_product)
    loss = exp_loss/(torch.sum(mask_positive.float())  + torch.sum(mask_negative.float()))
    return loss

map:

def mean_average_precision(database_codes,database_labels,test_codes,test_labels, R):
    query_num = test_codes.shape[0]

    sim = np.dot(database_codes, test_codes.T)
    ids = np.argsort(-sim, axis=0)
    APx = []

    for i in range(query_num):
        label = test_labels[i, :]
        label[label == 0] = -1
        idx = ids[:, i]
        imatch = np.sum(database_labels[idx[0:R], :] == label, axis=1) > 0
        relevant_num = np.sum(imatch)
        Lx = np.cumsum(imatch)
        Px = Lx.astype(float) / np.arange(1, R + 1, 1)
        if relevant_num != 0:
            APx.append(np.sum(Px * imatch) / relevant_num)

    return np.mean(np.array(APx))

[yuanli2333]

@yl2488 How did you train the model? I fine-tuned on the ResNet50, the mAP is 0.7780. This is my experiment details: Model: ResNet50(the activation function is tanh) Loss: pairwise cross-entropy(without the weight to account for the data imbalance, with alpha==1) R: 5000(dataset is nuswide_81) epoch: 50(if go on training, the loss will be lower)

loss:

def pairwise_loss(outputs1,outputs2,label1,label2):
    similarity = Variable(torch.mm(label1.data.float(), label2.data.float().t()) > 0).float()
    dot_product = torch.mm(outputs1, outputs2.t())
    #exp_product = torch.exp(dot_product)

    mask_positive = similarity.data > 0
    mask_negative = similarity.data <= 0

    exp_loss = torch.sum(torch.log(1+torch.exp(-torch.abs(dot_product))) + torch.max(dot_product, Variable(torch.FloatTensor([0.]).cuda()))-similarity * dot_product)

    #exp_loss = torch.sum(torch.log(1 + exp_product) - similarity * dot_product)
    loss = exp_loss/(torch.sum(mask_positive.float())  + torch.sum(mask_negative.float()))
    return loss

map:

def mean_average_precision(database_codes,database_labels,test_codes,test_labels, R):
    query_num = test_codes.shape[0]

    sim = np.dot(database_codes, test_codes.T)
    ids = np.argsort(-sim, axis=0)
    APx = []

    for i in range(query_num):
        label = test_labels[i, :]
        label[label == 0] = -1
        idx = ids[:, i]
        imatch = np.sum(database_labels[idx[0:R], :] == label, axis=1) > 0
        relevant_num = np.sum(imatch)
        Lx = np.cumsum(imatch)
        Px = Lx.astype(float) / np.arange(1, R + 1, 1)
        if relevant_num != 0:
            APx.append(np.sum(Px * imatch) / relevant_num)

    return np.mean(np.array(APx))

Hi, after fine-tuning, I also obtain similar result with you by using ResNet50. MAP@64bit, 77.51 for bus_wide