search

microsoft / Graphormer

Graphormer is a general-purpose deep learning backbone for molecular modeling.
MIT License
2.08k stars 334 forks source link

a question about code #18

Closed CAODH closed 3 years ago

CAODH commented 3 years ago

def convert_to_single_emb(x, offset=512): feature_num = x.size(1) if len(x.size()) > 1 else 1 feature_offset = 1 + \ torch.arange(0, feature_num * offset, offset, dtype=torch.long) x = x + feature_offset return x could you tell me why add this func? i am very about that? thanks!

CAODH commented 3 years ago

but why use add a different offset in different dim in a tensor ?maybe add same offset?

---Original--- From: "Chengxuan @.> Date: Mon, Aug 16, 2021 11:22 AM To: @.>; Cc: @.**@.>; Subject: Re: [microsoft/Graphormer] a question about code (#18)

Hi @CAODH , This is an implementation trick. We let different categories share a lookup table (E.g., nn.Embedding). Therefore, we add offsets to different column in order to separate them in the lookup table.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

zhengsx commented 3 years ago

For each dimension of category features, we assign (by default) 512 offsets to the index of lookup table, therefore, we shift each dimension for 512.

For example, an atom has 3 category features, and its feature vector looks like torch.Tensor([2, 1, 5]). Then the shifted feature vector would be something like torch.Tensor([2, 513, 1029]).

zhengsx commented 3 years ago

Close this issue due to inactivity for a long time. Feel free to reopen it if the problem still exist.

JeremySun1224 commented 5 months ago

Why is feature_offset incremented by 1? feature_offset = 1 + torch.arange(0, feature_num * offset, offset, dtype=torch.long)