Export model to tf serving

[luozhouyang]

Created a module exporter.py to export pre-trained model, and add a few extra arguments in nmt.py module which are needed for exporting. For testing exporter, I trained a simple model using test datas from the nmt/testdata/ folder, and saved the model in nmt/testdata/models. After this, run the test of exporter, it will generate a new folder named export in nmt/testdata/models, which is what we expected and can be served by tf serving.

[nidhidamodaran]

@luozhouyang , is the problem with output fixed yet ?

[hpulfc]

@nidhidamodaran @luozhouyang there is some issues about initialize the iterator. also the output maybe a node(op).

[bidai541]

@nidhidamodaran @JeffreyYun iterator is fixed while run initailization. Modify model_helper.py. Do not use tf.data.dataset API while inference.

def create_infer_model(model_creator, hparams, scope=None, extra_args=None):
  graph = tf.Graph()
  src_vocab_file = hparams.src_vocab_file
  tgt_vocab_file = hparams.tgt_vocab_file

  with graph.as_default(), tf.container(scope or "infer"):
    src_vocab_table, tgt_vocab_table = vocab_utils.create_vocab_tables(
        src_vocab_file, tgt_vocab_file, hparams.share_vocab)
    reverse_tgt_vocab_table = lookup_ops.index_to_string_table_from_file(
        tgt_vocab_file, default_value=vocab_utils.UNK)

    src_placeholder = tf.placeholder(shape=[None], dtype=tf.string)
    batch_size_placeholder = tf.constant(1, tf.int64)

    iterator = pre_process(
      src_placeholder,
        src_vocab_table,
        eos=hparams.eos,
        src_max_len=hparams.src_max_len_infer)
    model = model_creator(
        hparams,
        iterator=iterator,
        mode=tf.contrib.learn.ModeKeys.INFER,
        source_vocab_table=src_vocab_table,
        target_vocab_table=tgt_vocab_table,
        reverse_target_vocab_table=reverse_tgt_vocab_table,
        scope=scope,
        extra_args=extra_args)
  return InferModel(
      graph=graph,
      model=model,
      src_placeholder=src_placeholder,
      batch_size_placeholder=batch_size_placeholder,
      iterator=iterator)

def pre_process(src_string, src_vocab_table, eos, src_max_len=35):
  src_eos_id = tf.cast(src_vocab_table.lookup(tf.constant(eos)), tf.int32)
  src_string = tf.string_split([src_string]).values

  if src_max_len:
    src_string = src_string[:, src_max_len]
  src = tf.cast(src_vocab_table.lookup(src_string), tf.int32)
  src = tf.expand_dims(src, axis=0)
  src_len = tf.size(src)
  src_len = tf.expand_dims(src_len, axis=0)
  return BatchedInput(
    initializer=None,
    source=src,
    target_input=None,
    target_output=None,
    source_sequence_length=src_len,
    target_sequence_length=None)

[luckysofia]

Hi I want to load model from exported_model, but when I run, there is a bug like: FailedPreconditionError (see above for traceback): Attempting to use uninitialized value dynamic_seq2seq/decoder/output_projection/kernel

Traceback (most recent call last): File "/data/users/wangsu/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1292, in _do_call return fn(*args) File "/data/users/wangsu/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1277, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/data/users/wangsu/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1367, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value dynamic_seq2seq/decoder/output_projection/kernel [[{{node dynamic_seq2seq/decoder/output_projection/kernel/read}} = IdentityT=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

@luozhouyang How load model and infer from exported_model rather than .ckpt files ?