How to use pre-train model for paraphrase generation?

[GabrielLin]

It seems that these is no code relative to these function? Could you please show me the way? Thanks.

[iamaaditya]

Ya, I have not provided the code to only decode. I will soon.

[GabrielLin]

Waiting for you.

[Bachstelze]

is this just planned for english paraphrasing? I dont think there is a way for multilingual transfer learning though it would be very interesting for researchers in other languages to test the approach against their specific language characteristics.

[iamaaditya]

@GabrielLin Sorry for the wait. Were you able to do it? I can provide you pointers on how to do that. Easiest way would be to just pass the same trainer module with train=False.

[iamaaditya]

@Bachstelze I would be interested in multilingual paraphrase. That is certainly a research area. Email me if you are interested in working on this.

[miheir]

I am interested in decode only implementation as well. Also, it would be interesting to use this with other english and multilingual training sets, so a train data processing reference would be helpful.

[GabrielLin]

@iamaaditya Could you please show the detail code? I search all three .py files and cannot find a parameter named train.

[yuchenlin]

@GabrielLin How about just set learning rate as 0?

[ghost]

Hi, any update on this? I think it would be of great utility to have a paraphrase function

[Bachstelze]

Due to the fact that all code has been deleted, i think there will be no updates in this rep.

[ghost]

@Bachstelze I just downloaded the code, it is in the dev branch

[jendib]

Could you provide an example for text generation please?

[jren2017]

@GabrielLin Have you find the way to implement the decode process with the pretrained-model?

[FirestarterUA]

any progress with this?

[yuchenlin]

I think we can simply use OpenNMT for paraphrasing, which is very simple.

[ghost]

just simply use rslt = estimator.predict(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn)]) to get the prediction.

[FirestarterUA]

just simply use rslt = estimator.predict(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn)]) to get the prediction.

can you help with all code?

[khliu1238]

i think he meant to put the line in main function of train.py

[khliu1238]

still not working, the line:

rslt = estimator.predict(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn)])

return empty list with command

python train.py --use_residual_lstm

when loading pretrained model.

[khliu1238]

resolved. need a for loop to retrieve result, like the following:

def get_vocabulary(fn):
    f = open(fn, 'r')
    return f.read().splitlines()

vocab = get_vocabulary(FLAGS.vocab_filename)
rslt = estimator.predict(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn)]) 
for k in rslt:
    for i in k:
        print '%s ' % vocab[i],
    print('')

command:

python train.py

but the result doesn't look very good, nor is it outputting result as described on readme

[ghost]

@khliu1238 Sorry for the late reply. That's exactly what I mean. The reason of bad result maybe what it uses currently is greedy decoder. Replacing BeamSearchDecoder improves the performance.

[samarthsikand]

@khliu1238 I have put your estimator.predict code in the main function, but the output is printing infinitely. Is there any problem with the feed_fn or input_fn ?

` def main(args) :

  tf.logging._logger.setLevel(logging.INFO)

  data  = Data(FLAGS)
  model = Seq2seq(data.vocab_size, FLAGS)

  input_fn, feed_fn = data.make_input_fn()
  print_inputs = tf.train.LoggingTensorHook( ['source', 'target', 'predict'], every_n_iter=FLAGS.print_every, 
     formatter=data.get_formatter(['source', 'target', 'predict']))
  est = tf.estimator.Estimator(model_fn=model.make_graph, model_dir=FLAGS.model_dir, params=FLAGS)
  print("First here")
  #est.train(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn), print_inputs], steps=FLAGS.iterations)

  vocab = get_vocab(FLAGS.vocab_filename)
  pred = est.predict(input_fn=input_fn, hooks=[tf.train.FeedFnHook(feed_fn)])
  for k in pred:
      for i in k:
          print('%s '%vocab[i]),
      print('')

`

[khliu1238]

mine also print infinitely and I guess it's normal result. there's a infinite for loop in code.

[iamaaditya]

see https://github.com/iamaaditya/neural-paraphrase-generation/blob/dev/predict.py