Code about Noise-Aware-Masking

[fzb408]

Hello, thank you for your great work, may I ask what part of the implementation code of the Noise-Aware-Masking module, thank you

[WentaoTan]

In lines 79-83 of the /processor/processor.py file:

with torch.no_grad(): similarity_matrix = torch.einsum('nld,nkd->nlk', [F.normalize(fu_t_feats,dim=-1), F.normalize(fu_i_feats[:,1:,:],dim=-1)]) similarity_matrix = similarity_matrix.max(-1)[0] for idx, sim in zip(batch['image_ids'].data, similarity_matrix): trainset[idx][-1] = sim.data.cpu().numpy()

the code calculates the probability of each word token being masked. It updates the trainset by assigning the computed masking probabilities to the corresponding entries.

Then, in lines 53-56:

with torch.no_grad(): if epoch % 1 == 0: logger.info('Reconstruct the train loader') train_loader = build_filter_loader(args, trainset)

the refreshed trainset is used to reconstruct a new data loader. This data loader contains the masking probabilities computed from the previous epoch.

[fzb408]

Thank you very much for your reply!

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2024年5月17日(星期五) 下午4:44 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [WentaoTan/MLLM4Text-ReID] Code about Noise-Aware-Masking (Issue #2)

In lines 79-83 of the /processor/processor.py file:

with torch.no_grad(): similarity_matrix = torch.einsum('nld,nkd->nlk', [F.normalize(fu_t_feats,dim=-1), F.normalize(fu_i_feats[:,1:,:],dim=-1)]) similarity_matrix = similarity_matrix.max(-1)[0] for idx, sim in zip(batch['image_ids'].data, similarity_matrix): trainset[idx][-1] = sim.data.cpu().numpy()

the code calculates the probability of each word token being masked. It updates the trainset by assigning the computed masking probabilities to the corresponding entries.

Then, in lines 53-56:

with torch.no_grad(): if epoch % 1 == 0: logger.info('Reconstruct the train loader') train_loader = build_filter_loader(args, trainset)

the refreshed trainset is used to reconstruct a new data loader. This data loader contains the masking probabilities computed from the previous epoch.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>