Extend to Elmo embeddings

[ghaddarAbs]

Great work!!!!

I was wondering if its possible to replicate/extend the same work for Elmo embeddings

With tensorflow_hub, calling Elmo is not more complicated than:

def build_elmo():
    token_ph = tf.placeholder(tf.string, [None, None])
    len_ph = tf.placeholder(tf.int32, [None])
    elmo_module = hub.Module("https://tfhub.dev/google/elmo/2")
    lm_embeddings = elmo_module(inputs={"tokens": token_ph, "sequence_len": len_ph},
                                signature="tokens",
                                as_dict=True)

    word_emb = lm_embeddings["word_emb"]# [num_sentences, max_sentence_length, 512]
    lm_emb = tf.stack([tf.concat([word_emb, word_emb], -1),
                       lm_embeddings["lstm_outputs1"],
                       lm_embeddings["lstm_outputs2"]], -1) # [num_sentences, max_sentence_length, 1024, 3]

    return token_ph, len_ph, lm_emb

It is a suggestion more than an issue ..... feel free to close it :)

[yuchenlin]

+1 it would be very helpful to include other optional contextualized embedding model in such a powerful service tool

[yuchenlin]

Hi @ghaddarAbs , I was wondering if you're going to fork and implement this? Maybe we can work on this together; seems there are many such hub ready to use

[amansrivastava17]

I have developed a repo https://github.com/amansrivastava17/embedding-as-service having support for most of the embeddings