chingyaoc / debias_vl

Code for Debiasing Vision-Language Models via Biased Prompts
MIT License
53 stars 4 forks source link

is code equivalent to the formulas in the paper? #4

Closed ellemcfarlane closed 1 month ago

ellemcfarlane commented 1 month ago

trying to confirm whether (code link):

def get_A(z_i, z_j):
    z_i = z_i[:, None]
    z_j = z_j[:, None]
    return (np.matmul(z_i, z_i.T) + np.matmul(z_j, z_j.T) - np.matmul(z_i, z_j.T) - np.matmul(z_j, z_i.T))

def get_M(embeddings, S):
    d = embeddings.shape[1]
    M = np.zeros((d, d))
    for s in S:
        M  += get_A(embeddings[s[0]], embeddings[s[1]])
    return M / len(S)

M = get_M(candidate_embeddings, S)
G = args.lam * M + np.eye(M.shape[0]) 
P = np.linalg.inv(G)

is nearly mathematically equivalent to the orthogonal projection matrix formula from the paper (particularly the A and M calculations):

Captura de pantalla 2024-10-08 a las 14 44 23
ellemcfarlane commented 1 month ago

Ah I see now it's equivalent to the closed-form calibrated projection matrix:

Captura de pantalla 2024-10-08 a las 14 51 50