How to export?

[jinamshah]

I needed to know how to write the serving function to export the trained xlnet model. I have this right now:

def serving_input_fn(): with tf.variable_scope("model"): feature_spec = { "input_ids": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "input_mask": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "segment_ids": tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64), "label_ids": tf.FixedLenFeature([], tf.int64), } serialized_tf_example = tf.placeholder(dtype=tf.string, shape=[None], name='input_example_tensor') receiver_tensors = {'examples': serialized_tf_example} features = tf.parse_example(serialized_tf_example, feature_spec) return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

EXPORT_DIR = 'gs://{}/export/{}'.format(BUCKET, TASK_VERSION) estimator._export_to_tpu = False # this is important path = estimator.export_savedmodel(EXPORT_DIR, serving_input_fn)

This is throwing me errors. Please note: this is the function that I used for Bert, and as I am no expert in tensorflow, I don't understand why it won't work. It throws a type mismatch error

[lukemelas]

Hello @jinamshah , have you resolved this issue? If not, let me know and I can help debug.

[jinamshah]

Hey @lukemelas I was able to resolve this issue. I wrote my own function to help me do this. Thanks!

[lukemelas]

Great!

[lukemelas]

I believe this issue can be closed.

[ashgorithm]

I was also getting type mismatch errors while exporting model.

TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [int32, int64] that don't all match.

How did you resolve this issue?

[jinamshah]

@ashgorithm
for name in list(features.keys()): t = features[name] if t.dtype == tf.int64: t = tf.cast(t, tf.int32) features[name] = t I believe this would help you. Basically, the input is required to be int32

[AndrewPelton]

@jinamshah would you be able to share your solution? I am having the same problem

[hexiaoyupku]

@jinamshah would you be able to share your solution? I am having the same problem

+1, please!

[kobkrit]

@jinamshah would you be able to share your solution? I am having the same problem +1 Please!!

[kobkrit]

Hey @AndrewPelton @hexiaoyupku I solved it :) Thank you to @jinamshah

  def serving_input_fn():
        with tf.variable_scope("foo"):
          feature_spec = {
              "unique_ids": tf.FixedLenFeature([], tf.int64),
              "input_ids": tf.FixedLenFeature([FLAGS.max_seq_length], tf.int64),
              "input_mask": tf.FixedLenFeature([FLAGS.max_seq_length], tf.float32),
              "segment_ids": tf.FixedLenFeature([FLAGS.max_seq_length], tf.int64),
              "cls_index": tf.FixedLenFeature([], tf.int64),
              "p_mask": tf.FixedLenFeature([FLAGS.max_seq_length], tf.float32)
          }

          serialized_tf_example = tf.placeholder(dtype=tf.string,
                                                 shape=[FLAGS.predict_batch_size], #[None],
                                                 name='input_example_tensor')

          receiver_tensors = {'examples': serialized_tf_example}
          features = tf.parse_example(serialized_tf_example, feature_spec)

          for name in list(features.keys()): 
            t = features[name] 
            if t.dtype == tf.int64:
                t = tf.cast(t, tf.int32)
                features[name] = t

          return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

  if FLAGS.do_export:
    estimator._export_to_tpu = False  # this is important
    print("Estimator save model..")
    estimator.export_savedmodel('export_t', serving_input_fn)

[rob-nn]

Could someone please send me an example of http post request to this serving? thanks!