Distance range for training with softmax?

[aytackocaman]

From Facenet paper it says: A distance of 0.0 means the faces are identical, 4.0 corresponds to the opposite spectrum, two different identities.

I think it is valid for training with triplet loss. Is it also valid for training softmax? If not what is the range?

Thanks.

[Sherry40931]

I think the distance range is coming from the L2-normalized embeddings but not related with the loss function.

From: this page we can see that the squared L2-distance and the cosine distance has linear relationship. Since -1 < cosθ < 1, we can calculate that 0 < squared_L2 < 4.

So I think the range is also valid for softmax and any other loss functions as long as you normalized the embeddings.

[amankhandelia]

Hi @Sherry40931

I just came across this link on stats.stackexchange which basically substantiates what you have commented above.

But if you look at the this snippet of code (also written below for ease) used by David for calculating the distance, it does not have that property. If you may, can you please explain the relationship between the two and is there a situation where we should use the method mentioned in the above link.

def distance(embeddings1, embeddings2, distance_metric=0):
    if distance_metric==0:
        # Euclidian distance
        diff = np.subtract(embeddings1, embeddings2)
        dist = np.sum(np.square(diff),1)
    elif distance_metric==1:
        # Distance based on cosine similarity
        dot = np.sum(np.multiply(embeddings1, embeddings2), axis=1)
        norm = np.linalg.norm(embeddings1, axis=1) * np.linalg.norm(embeddings2, axis=1)
        similarity = dot / norm
        dist = np.arccos(similarity) / math.pi
    else:
        raise 'Undefined distance metric %d' % distance_metric 

    return dist

[Sherry40931]

Hi @amankhandelia I did not quite understand your question. But as the code is written, it has two types of distance metrics because the embeddings are normalized. Therefore it can either be calculated for euclidian distance or cosine similarity.

As when we should use the equation to transfer one metric to another, one occasion is that some paper (like the original paper of facenet) found the best threshold in euclidian distance and used it to calculate the accuracy, if you want to use that value with cosine similarity metric, then you need to calculate the threshold in cosine metric by yourself. I can not think of another example but I think the equation might be useful if you are developing project related to the threshold of face verification.

Hope I answered your question!