Minimum risk training

[jindrahelcl]

Implement minimum risk trainer as described in http://arxiv.org/abs/1512.02433.

• sampling is done by computing top-k translations (beam search is already done)

[jindrahelcl]

starting to work on this, very slowly though..

[tomasmcz]

@jindrahelcl: What do you mean by “beamsearch is already done”?

[jindrahelcl]

@tomasmcz that's what @jlibovicky says. Anyway, the implementation of beamsearch could be used for sampling the sentences for estimating the risk.

[tomasmcz]

In that case, beamsearch is done, but it is an ugly hack and I'm not sure it's working correctly right now (see #5).

[StoyanVenDimitrov]

Hello team,

I start working on the min risk training as an university project. I'll let you know about my progress here.

Best regards: Stoyan

[StoyanVenDimitrov]

Hi,

what is the easiest way in the decoder to access the words from the vocabulary while training given the IDs? And actually I need them postproccessed since I want to build a whole sentence from the samples, that I'll compute BLEU score on.

Thank you and best regards: Stoyan

[jlibovicky]

Hi, the decoder has a vocabulary as a property, so you can access it and words for indices. However, I don't think you should do this in decoder. Decoder is a description of the model part used for decoding, the actual work is done in runners and trainers.

In this case, you probably should create a new trainer for minimum risk training and do the sentence handling there. You have more options how you can score the sentences

• write an in-graph approximation of BLEU score (something as Google's GLEU score)
• write the sentence evaluation as a py_func
• call the session twice: first decode the sentences, compute the BLEU score in the body of your trainer's executable and than let the executable run again (this is a build-in functionality of the tf_manager)

[StoyanVenDimitrov]

Hi,

thank you! I was thinking of a mrt_loss as a function in the decoder directly, like the tf...sequence_loss you use for xent. I see your point on not doing it this way. Creating a new trainer should means to create a class similar to the cross_entropy_trainer, where first a runner is called to obtain the sentences. The runner should do the same like the GreedyRunExecutable, but with a decoder that samples from the runtime_logprobs it obtained and I'll also add a fetche for mrt_loss. Don't sure how to get the real sentences out of the runner though. After that I thought I can use the neuralmonkey.evaluators.bleu.BLEUEvaluator to obtain the score. The rest of the computations needed for a MRT will be done in a function like xent_objective that returns Objective object. That means also, that I have to overwrite add a runner to the Objective.

What do you think? Sorry, I'm confused with the whole classes.

Best, Stoyan

[jlibovicky]

You're basically right. Your trainer needs to have an executable that will first do the inference including the postprocessing and evaluation. At this point, however, the executable will not yield any result. It the next step it will finally prepare the the fetch that will compute the MRT loss and an additional feed dict for a placeholder with BLEU. The additional parts of the computational graph (including the BLEU placeholder) should be defined in the trainer's constructor.

You should be also careful with the evaluators. BLEUEvaluator computes corpus-level BLEU which includes the brevity penalty over the whole corpus. What you need to compute is sentence-level BLEU. NLTK implements it, so you can find and inspiration there.

[StoyanVenDimitrov]

I have now a runner that can do the postprocressing and gleu and also gives me the prob of the sentences, did it only to test how the parts will work. I'll try to transffer what I've done there into a trainer. Is there an easy way like running this runner in the new trainer first (), taking the outputs of the runner and computing a MRT loss and then adding the loss to Objective?

[StoyanVenDimitrov]

What do you exactly mean with "It the next step it will finally prepare the the fetch that will compute the MRT loss and an additional feed dict for a placeholder with BLEU"?

[jlibovicky]

Every instance of runner or trainer can five you an instance of Executable that is execute in the tf_manager which executes its fetches in a loop until the result attribute of the Executable is None. In this case, you will probably need to run some fetches twice. In the first step of the tf_manager's loop, you will get the indices and in the collect_results method, you will get the actual strings, and compute BLEU or whatever metric you like (this you said you have implemented). Once you have the score, you can get compute the objective, but you need to feed the BLEU value into the graph somehow. For that, you can use a tf.placeholder initialized in the __init__ method of your trainer.

You can probably inherit your trainer from GenericTrainer and reuse some of its parts, mostly the __init__ method, but you can't reuse TrainExecutable, you need to write a completely new one, in fact with functionality of the executable class from the GreedyRunner and from the GenericTrainer.

If you have your code in a fork on GitHub, I can eventually have a look at the code, if you want.

[StoyanVenDimitrov]

Thank you! It works now, except that I couldn't find out how to feed the score to the placeholder, since the session execution happens somewhere else. Do you have any suggestions on how to do it? Best, Stoyan

[jindrahelcl]

Hi,

as long as the runner sets its result, the tf_manager will keep calling its next_to_execute method and run the session again with the provided executable. Look at the code in tf_manager, I think this property can be used for this..

Best, Jindra

Dne 16. 4. 2017 12:15 napsal uživatel "StoyanVenDimitrov" < notifications@github.com>:

Thank you! It works now, except that I couldn't find out how to feed the score to the placeholder, since the session execution happens somewhere else. Do you have any suggestions on how to do it? Best, Stoyan

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufal/neuralmonkey/issues/2#issuecomment-294344209, or mute the thread https://github.com/notifications/unsubscribe-auth/ABwcs5Kew8WmOllNmDePqlxU52ytpbaVks5rwepZgaJpZM4I6qKv .

[StoyanVenDimitrov]

Hi, here is my code https://github.com/StoyanVenDimitrov/nm011/blob/master/neuralmonkey/trainers/mrt_trainer.py But it consumes huge amount of memory. It will be great if you can see the reason for it. Because I wasn't able to optimize it.

Best, Stoyan