Neural label embeddings

OctoberChang / X-Transformer

X-Transformer: Taming Pretrained Transformers for eXtreme Multi-label Text Classification

BSD 3-Clause "New" or "Revised" License

135 stars 28 forks source link

Neural label embeddings #13

Closed vinaysetty closed 3 years ago

vinaysetty commented 3 years ago

Hi,

I am a bit unsure about how you created the neural label embeddings using XLNet or Roberta i.e, how are the files X.trn.finetune.xlnet.npy and Y.trn.finetune.xlnet.npy generated? I tried giving pifa-neural option in run_preprocess_label.sh but I get the error: FileNotFoundError: [Errno 2] No such file or directory: 'X.trn.finetune.xlnet.npy'

Any idea what I am missing?

Thanks!

OctoberChang commented 3 years ago

It's PIFA embedding that takes the fine-tuned instance embeddings as input, instead of the sparse TFIDF instance vector. See the PIFA constructor for more details (https://github.com/amzn/pecos/blob/mainline/pecos/xmc/base.py#L1519).