search

Aloriosa / GEMFormer

Global Explicit Memory Transformer
https://arxiv.org/abs/2311.18151
Apache License 2.0
0 stars 0 forks source link

ask for question #1

Open gongchuanyang opened 1 week ago

gongchuanyang commented 1 week ago

Your work is very meaningful, but I can't find the specific implementation of the function "QuestClsQASpanSuppClsFullDocModelOutput" in modeling_gemformer.py

Aloriosa commented 1 week ago

@gongchuanyang Thank you for noticing the bug. The "QuestClsQASpanSuppClsFullDocModelOutput" is the model output formatting class. PLease see the updated modeling_gemformer.py

gongchuanyang commented 1 week ago

Thank you

gongchuanyang commented 6 days ago

When the program is running, in modeling_gemformer.py file def or_softmax_cross_entropy_loss_one_doc(logits, target, ignore_index=-1, dim=-1, reduction='sum'): """ from the Longformer code: loss function suggested in section 2.2 here https://arxiv.org/pdf/1710.10723.pdf """ assert logits.ndim == 2, f"{logits.shape}, {logits}" assert target.ndim == 2 assert logits.size(0) == target.size(0) The assert logits.size(0) == target.size(0) in will show ValueError: Expected input batch_size (40) to match target batch_size (10). Error

Aloriosa commented 3 days ago

Hello, @gongchuanyang! Sorry for not getting back to you sooner, my notifications didn't work. It seems like assert logits.size(0) == target.size(0) is not the cause of the error if it says ValueError. The error says about the input-output dimensions mismatch so I suggest you to re-check the dimensions somewhere before the loss calculation. Re-opened the issue.