search

allenai / allennlp

An open-source NLP research library, built on PyTorch.
http://www.allennlp.org
Apache License 2.0
11.76k stars 2.25k forks source link

Example use of bidirectional_lm? #2012

Closed yfeng997 closed 6 years ago

yfeng997 commented 6 years ago

I am trying to use the model bidirectional_lm, but was not able to find pre-trained weight file. I am also not exactly sure how to train this model on my own.

It would be great if we can have a pre-trained model for bidirectional_lm. For example, for semantic role labeling model, we have "https://s3-us-west-2.amazonaws.com/allennlp/models/srl-model-2018.05.25.tar.gz", as shown on AllenNLP website.

I have considered training the model on my own, but training will take considerably more effort given that I don't have a GPU, whereas I am really just trying to get a proof-of-concept. I have also checked out the TensorFlow implementation of BiLM. But my mac does not have a GPU, whereas the README on this repository says

this repository requires GPUs.

I am also unfamiliar with TensorFlow, so I am looking mostly for a PyTorch implementation of this model.

As mentioned in this Google Group post, I am trying to access the probability prediction for each word, given the contextual words both before and after the target word. If there are any other convenient way of accessing them, that would also be great.

Thanks for any help!

schmmd commented 6 years ago

@YuansongFeng the language model is a bit different than our other models, so it's not listed on https://allennlp.org/models. You should find the information you're looking for here though: https://allennlp.org/elmo