Converting an AllenNLP model for android mobile

[cy-eom]

System (please complete the following information):

• OS: Linux (Ubuntu 18.04)
• Python version: 3.7
• AllenNLP version: 0.8.3
• PyTorch version: 1.3.1

Hi all,

I'm looking for a way to convert an AllenNLP model using Pytorch mobile library, to run the model on an android mobile.

We implemented our model using BiLSTM-CNN-CRF algorithm, and after the training step, results are archived as model.tar.gz file, including weight.th, config.json, and a sort of vocabulary text files.

Below is the structure of our model object.

after  load_archive: archive Archive(model=CrfTagger(
  (text_field_embedder): BasicTextFieldEmbedder(
    (token_embedder_pos_tags): Embedding()
    (token_embedder_token_characters): TokenCharactersEncoder(
      (_embedding): TimeDistributed(
        (_module): Embedding()
      )
      (_encoder): TimeDistributed(
        (_module): CnnEncoder(
          (_activation): ReLU()
          (conv_layer_0): Conv1d(30, 30, kernel_size=(3,), stride=(1,))
        )
      )
    )
    (token_embedder_tokens): Embedding()
  )
  (encoder): PytorchSeq2SeqWrapper(
    (_module): LSTM(335, 100, batch_first=True, bidirectional=True)
  )
  (dropout): Dropout(p=0.5, inplace=False)
  (tag_projection_layer): TimeDistributed(
    (_module): Linear(in_features=200, out_features=3, bias=True)
  )
  (crf): ConditionalRandomField()
), config=<allennlp.common.params.Params object at 0x7f9219bb5f50>)

BTW, I found a few hints from pytorch mobile - android and tutorial, and they explain that torch.jit.save creates a .pt file, a converted pytorch model.

For example, I'm thinking of using HelloWorld app where serialized model is used, and It provides a python script in the root folder of HelloWorld app:

import torch
import torchvision

model = torchvision.models.resnet18(pretrained=True)
model.eval()
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("app/src/main/assets/model.pt")

However, for me still it is not clear that whether this approach is also useful to AllenNLP model too, because I think the structure of our model is not that same as general model used in above example.

Could anyone tell me some useful approaches or code sniffet that would work effectively?

I hope your kind responses. Thanks.

[dirkgr]

I'm not super familiar with how torch runs on mobile, but my impression is that you have to get your whole model to run under TorchScript. In your example, you're calling torch.jit.trace, but tracing only works in a subset of cases. There is no guarantee that any of our models work in this case.

I know some people are looking into making the whole thing compatible with TorchScript, but I can't give you a timeline for that.

[schmmd]

Closing as this isn't a priority for us presently. If someone is interested in working on this and contributing for TorchScript, please follow up on a separate issue and we can discuss.