unk replacement with corresponding words in source input in the output sentences when using --predict_from_input

[liperrino]

please i would like to replace the unk in the output sentences with the corresponding words in the source sentence when using the option: --predict_from_input. Meaning going from german to english, if a word was seen as unk in the provided sentence it should appear exactly in the output sentence at the good position.

[lukemelas]

Hello,

Thanks for your interest in the repo.

If you do not want in your predictions, you can manually block the network from producing the character using the remove_tokens argument in model.predict_k. For example, you can do exactly what is done on line 21 before line 43 in training/predict.py

[liperrino]

Thanks a lot for the proposition. But to be sure we are talking about the same thing, let us considerate that we are making a traduction from English to French. Under is a better explanation of what i want.

INPUT SENTENCE : i am going to school tomorrow with my teacher. INPUT USE BY predict_from_input : i am going to school with my OUTPUT RESULT : je vais à l'école avec mon WHAT I WANT AS RESULT : je vais à l'école tomorrow avec mon teacher

[liperrino]

Thanks a lot for the proposition. But to be sure we are talking about the same thing, let us considerate that we are making a traduction from English to French. Under is a better explanation of what i want.

INPUT SENTENCE : i am going to school tomorrow with my teacher. INPUT USE BY predict_from_input : i am going to school with my OUTPUT RESULT : je vais à l'école avec mon WHAT I WANT AS RESULT : je vais à l'école tomorrow avec mon teacher

INPUT SENTENCE : i am going to school tomorrow with my teacher. INPUT USE BY predict_frominput : i am going to school unk with my unk RESULT : je vais à l'école unk avec mon unk_

WHAT I WANT AS RESULT : je vais à l'école tomorrow avec mon teacher

[liperrino]

I have train the model using the default parameters but when doing the prediction like in your post: , i obtain a different result.

German: Ihre erwies sich als komplizierter als gedacht . 1: Her were willing to think more than more than exciting . 2: Her turned to themselves more complicated than exciting . 3: Her were willing to think more than more than anticipated . 4: Her were willing to think more than harder than thought . 5: Her were willing to think more than more than complex than most . Finished in 0.3209352493286133

[liperrino]

Is something wrong with the model: I used this input: Wir bringen Kinder zur welt . I obtained this results:

German: Wir bringen Kinder zur 1: We bring kids to children to children to the children children to introduce children to children imaging . We bring children to pet . 2: We bring kids to children to children to the children children to introduce children to children imaging . We gather kids to children to children 3: We bring kids to children to children to the children children to introduce children to children imaging . We gather children to children to children 4: We bring kids to children to children to the children children to introduce children to children imaging . We bring children to pet , 5: We bring kids to children to children to the children children to introduce children to children imaging . We bring children to pet . We bring

[liperrino]

I have try to use GRU instead of LSTM and i am having errors. Please try to check it for me.

[liperrino]

Happy New Year. Please how can i used GRU instead of LSTM in your code?

[lukemelas]

Hello,

Apologies for the late reply. I am currently traveling, but I will absolutely respond to you about using GRUs when I return to school.

Best, Luke

On Sun, Jan 13, 2019 at 10:02 AM liperrino notifications@github.com wrote:

Happy New Year. Please how can i used GRU instead of LSTM in your code?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lukemelas/Machine-Translation/issues/1#issuecomment-453851749, or mute the thread https://github.com/notifications/unsubscribe-auth/AMsOMCKd3qMcbzVfQZDzdMQW4fmbzPq9ks5vC3S3gaJpZM4ZgmMC .

[liperrino]

Thanks a lot

[lukemelas]

Just got back to school!

To switch from LSTM to GRU, try the following:

• Change line 17 in models/Decoder.pyand line 18 in models/Encoder.py from LSTM to GRU
• Change lines 27-30 in models/Encoder.pysuch that h0is a single tensor (i.e. init) rather than a tuple of tensors (i.e. (init, init.clone())).

Let me know how it works for you!

[liperrino]

Thank you very much. I will definetly try it and give you feedback.

[liperrino]

Please like i tall you before i would like to change your code to get something like this:

INPUT SENTENCE : i am going to school tomorrow with my teacher. INPUT USE BY predict_from_input : i am going to school unk with my unk RESULT : je vais à l'école unk avec mon unk

WHAT I WANT AS RESULT : je vais à l'école tomorrow avec mon teacher

Please how can i do this?

[liperrino]

Hello. I have change it like you have said for GRU. But i get nan concerning loss after epoch 1. Here is the output:

Loaded embedding vectors from np files Weight tying! Validation complete. BLEU: 0.000 New best: 0.000 Validation time: 115.180 PPL: 1.000

Epoch [1/50] Batch [10/1861] Loss: 7.769

Epoch [1/50] Batch [1010/1861] Loss: 4.505

Epoch [1/50] complete. Loss: nan Training time: 47292.040 Validation complete. BLEU: 0.000 Validation time: 84.113 PPL: 1.000 Epoch [2/50] Batch [10/1861] Loss: nan Epoch [2/50] Batch [1010/1861] Loss: nan Epoch [2/50] complete. Loss: nan Training time: 6695.366 Validation complete. BLEU: 0.000 Validation time: 92.769 PPL: 1.000 Epoch [3/50] Batch [10/1861] Loss: nan Epoch [3/50] Batch [1010/1861] Loss: nan Epoch [3/50] complete. Loss: nan Training time: 5903.140 Validation complete. BLEU: 0.000 Validation time: 89.201 PPL: 1.000 Epoch [4/50] Batch [10/1861] Loss: nan

[liperrino]

According to the results of your work done, i noticed that when phrase is entered in a source language (ex : German), the output results (phrases) is processed in a target language (ex : English), my question goes this way ; « Is the processed result based on translation only ? prediction only ? Or translation while taking into consideration some prediction ??

[liperrino]

I have so many question concerning your work. But i am afraid to disturb you. I don't know if i should expose them to you or not.

[liperrino]

I would like to try:

• teacher forcing
• label smoothing
• attention
• transformer

Please guide me to do this.

[liperrino]

I have made a mistake you have already do attention

[lukemelas]

Hello,

Yes, attention is already implemented :)

I must say, however, this repository is meant to be more of a simple example of sequence-to-sequence machine translation than a platform for implementing state-of-the-art models. If you are looking to explore all the super interesting ideas you mentioned above, I highly recommend checking out either OpenNMT or fairseq:

https://github.com/OpenNMT/OpenNMT-py https://github.com/pytorch/fairseq

These are both actively maintained, highly flexible frameworks for conducting cutting-edge research :)

On Fri, Mar 8, 2019 at 10:04 PM liperrino notifications@github.com wrote:

I have made a mistake you have already do attention

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lukemelas/Machine-Translation/issues/1#issuecomment-471140196, or mute the thread https://github.com/notifications/unsubscribe-auth/AMsOMMUMVeLTV5A5K2JrQs_NkQ51WsGYks5vUyTZgaJpZM4ZgmMC .

[liperrino]

Thanks a lot

[liperrino]

Hi. Please i would like to add some features i have read on beam search. Like coverage penalty and length normalization. But i don't know where to start. Can you help please?

[liperrino]

Hi. Please i still have some problems with the code. I got nan for the lost when training it. Loaded data. |TRG| = 11560. Time: 65.22. Loaded embedding vectors from np files Weight tying! /home/vivien/Bureau/NMT/Look/Machine-Translation-master/models/Seq2seq.py:90: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead. last_word_input = Variable(torch.LongTensor([last_word]), volatile=True).view(1,1) Validation complete. BLEU: 0.000 New best: 0.000 Validation time: 75.053 PPL: 1.000 Epoch [1/25] Batch [10/1861] Loss: 7.669 Epoch [1/25] Batch [1010/1861] Loss: nan

[AungZinPyae]

Please like i tall you before i would like to change your code to get something like this:

INPUT SENTENCE : i am going to school tomorrow with my teacher. INPUT USE BY predict_from_input : i am going to school unk with my unk RESULT : je vais à l'école unk avec mon unk

WHAT I WANT AS RESULT : je vais à l'école tomorrow avec mon teacher

Please how can i do this?

Have you found the solution for that?