search

adrinta / MAGNET

MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network
55 stars 11 forks source link

Can MAGNET be used for Extreme Multi-label classification? #3

Closed Owaiskhan9654 closed 3 years ago

Owaiskhan9654 commented 3 years ago

I read your paper and found it interesting. Now I am having a doubt that can we use MAGNET for Extreme Multi-label classification tasks? In paper you have only used few labels demonstrated ie. For Reuters example 90 categories, for RCV1-V2 only 103 Topics, for Slashdot only 291 classes. ??

Also with the BERT embeddings I can help you with that.. Instead of BERT we can use Embeddings from sentence transformers models. We can schedule a google meeting and discuss it there if possible. My mail owais96_scs@jnu.ac.in.

adrinta commented 3 years ago

@Owaiskhan9654 Thank you for your question. Firstly, it's not my paper or research but i try to reproduce the author work. I've tried to reach the author and ask some question or even asking their code. But he said that the research was implemented on Tensorflow 1 so the code is not reliable anymore so he said that i can reproduce the code by following this https://github.com/Megvii-Nanjing/ML-GCN and change the GCN to the GAT. But desperately i can't reproduce the best result by using BERT Embedding. With GloVe Embedding i think i get similar result (you can look at Bar Chart on the paper on Figure 2).

About your question, as mentioned in the paper that the paper said the model will be difficult to train because large number of label means the model need large number of correlation matrix. It will need very much resource of computation. If you think about hundred thousand of labels, i think it impossible to do that without very much resource like RAM.

Also i've tried Bert Embedding (Word Embedding not sentence ) like on this https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/ but i am not getting result like in the paper, the result even worse than GloVe Embedding. If you have another way to use BERT as Embedding, please let me know.

About discussion on google meet, maybe i have to look for my spare time.

Lastly, i am confused about text preprocessing too, because they're only mentioned little on text preprocessing so i tried text cleaning by my way.

Thank you

if you have another question or unclear explanation from me. please let me know.