Bad results of a simple translation task. Ask for help...:(

[JillinJia]

Actually, I plan to do a machine translation project with this seq2seq module. Before that, I just did a simple test and got a very bad result. I don't know where goes wrong. Pls help me... Here's the process:

#1. traning set

def generate_sequence(length, n_unique):
    return [randint(1, n_unique-1) for _ in range(length)]
x = np.array(generate_sequence(100000,100)).reshape(10000,10)
y = np.array(generate_sequence(50000,100)).reshape(10000,5)
x_encoder_input_data = to_categorical(x)
y_decoder_target_data = to_categorical(y)

#x_encoder_input_data.shape = (10000, 10, 100)
#10000 training data, x_input_length=10,x_input_dim=100
#y_decoder_target_data.shape = (10000, 5, 100)

#2. building&training model

model = SimpleSeq2Seq(input_dim=100,
             input_length=10, 
            output_dim=100, 
            hidden_dim=128,
            output_length=5)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
model.fit(x_encoder_input_data,  y_decoder_target_data, batch_size=128, epochs=500)

#3. the partial losses

Epoch 1/500
10000/10000  - 44s 4ms/step - loss: 7.2350 - acc: 0.0108
Epoch 2/500
10000/10000  - 20s 2ms/step - loss: 6.9500 - acc: 0.0104
Epoch 3/500
10000/10000  - 22s 2ms/step - loss: 10.7297 - acc: 0.0103
Epoch 4/500
10000/10000  - 20s 2ms/step - loss: 8.6834 - acc: 0.0096
Epoch 5/500
10000/10000  - 21s 2ms/step - loss: 8.4943 - acc: 0.0099
Epoch 6/500
10000/10000  - 19s 2ms/step - loss: 8.4487 - acc: 0.0100
Epoch 7/500
10000/10000  - 19s 2ms/step - loss: 8.6318 - acc: 0.0099
Epoch 8/500
10000/10000  - 18s 2ms/step - loss: 8.5765 - acc: 0.0099
Epoch 9/500
10000/10000  - 20s 2ms/step - loss: 8.4753 - acc: 0.0099
Epoch 10/500
10000/10000  - 20s 2ms/step - loss: 8.3738 - acc: 0.0099
Epoch 11/500
10000/10000  - 20s 2ms/step - loss: 8.3999 - acc: 0.0098
Epoch 12/500
10000/10000  - 19s 2ms/step - loss: 8.3108 - acc: 0.0099
Epoch 13/500
10000/10000  - 19s 2ms/step - loss: 8.3457 - acc: 0.0099
Epoch 14/500
10000/10000  - 19s 2ms/step - loss: 8.4852 - acc: 0.0098
Epoch 15/500
10000/10000  - 18s 2ms/step - loss: 8.4749 - acc: 0.0099
Epoch 16/500
10000/10000  - 18s 2ms/step - loss: 8.5881 - acc: 0.0098
Epoch 17/500
10000/10000  - 19s 2ms/step - loss: 8.3868 - acc: 0.0099
Epoch 18/500
10000/10000   - 21s 2ms/step - loss: 8.2499 - acc: 0.0098
Epoch 19/500
10000/10000   - 20s 2ms/step - loss: 8.4659 - acc: 0.0099
Epoch 20/500
10000/10000  - 20s 2ms/step - loss: 7.8421 - acc: 0.0099
Epoch 21/500
10000/10000   - 21s 2ms/step - loss: 7.6197 - acc: 0.0099
Epoch 22/500
10000/10000  - 20s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 23/500
10000/10000   - 19s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 24/500
10000/10000   - 21s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 25/500
10000/10000   - 19s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 26/500
10000/10000  - 22s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 27/500
10000/10000  - 22s 2ms/step - loss: 7.6193 - acc: 0.0099
··· ···

#4. predicting

for seq_index in range(6):
    predictions = model.predict(x_encoder_input_data[seq_index:seq_index+1])
    predicted_list=[]

    for prediction_vector in predictions:
        for pred in prediction_vector:
            next_token = np.argmax(pred)
            predicted_list.append(next_token)

    print('-')
    print('Input sentence:', X[seq_index])
    print('Decoded sentence:', predicted_list)
    print('Target sentence:', y[seq_index])

#5. the predicting results:

-
Input sentence: [28, 2, 46, 12, 21, 6]      #  x
Decoded sentence: [78, 78, 78, 78, 66] # y_predict
Target sentence: [82 22 82 41 27]          # y
-
Input sentence: [12, 20, 45, 28, 18, 42]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [43 36 30 13 64]
-
Input sentence: [3, 43, 45, 4, 33, 27]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [90 20 56 23 32]
-
Input sentence: [34, 50, 21, 20, 11, 6]
Decoded sentence: [78, 78, 78, 78, 66]
Target sentence: [27 57 50 57 81]
-
Input sentence: [47, 42, 14, 2, 31, 6]
Decoded sentence: [78, 78, 78, 78, 66]
Target sentence: [77 94 47 26 67]
-
Input sentence: [20, 24, 34, 31, 37, 25]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [11 48 99 67 66]