search

jrzaurin / pytorch-widedeep

A flexible package for multimodal-deep-learning to combine tabular data with text and images using Wide and Deep models in Pytorch
Apache License 2.0
1.25k stars 185 forks source link

how to implement target attention in this framework #208

Open LeiShenVictoria opened 3 weeks ago

LeiShenVictoria commented 3 weeks ago

In DeepInterestNetwork, there is a target attention between a candidate feature (one column) and a sequence feature, how to implement this target attention in this repo, which can be considered as an attention between a column in deep part (candidate) and the text part (sequence) i guess... Thanks a lot

jrzaurin commented 2 weeks ago

Hey @LeiShenVictoria

So I have not read the Deep Interest Network paper, I will, maybe I can incorporate some ideas to the library.

As of right now, the only thing "kind-of" similar you would have here are the attention weights of the models.

All model components that are based on attention mechanisms have an attribute called attention_weights: see here

I will have a look to the paper Deep Interest Network paper asap and see if I can come up with a quick answer that is more helpful :)

LeiShenVictoria commented 1 week ago

Hi, thanks for your reply. One more question is that how to implement the embedding-sharing operation for a candidate feature and a sequence feature.

jrzaurin commented 1 week ago

Hey @LeiShenVictoria

I would have to read the paper :)

I am busy at work now, but ill see what I can do asap