TypeError: unsupported format string passed to tuple.__format__

[hurlenko]

I'm trying to train the model using my own data. I used Dockerfile3.cpu as a step-by-step installation guide so I'm running latest master version with python3. Then I replaced

• data/corpora_processed/train_processed_dialogs.txt.
• data/corpora_processed/val_processed_dialogs.txt
• data/quality/context_free_validation_set.txt
• data/quality/context_free_questions.txt
• data/quality/context_free_test_set.txt with my own data. All my messages in all chats have neutral condition. After running python tools/prepare_index_files.py I got a c_idx_processed_dialogs.json file with the following content:

  {"0": "neutral"}

  Then I run python tools/train.py and it fails with the following error:

Log output:

```python [23.01.2019 17:40:37.133][INFO][11346][cakechat.utils.files_utils][91] Creating /home/unnamed/Projects/ml_chatbot/cakechat/data/tensorboard/steps [23.01.2019 17:40:37.134][INFO][11346][cakechat.tools/train.py][102] THEANO_FLAGS: floatX=float32,device=cpu [23.01.2019 17:40:37.141][INFO][11346][cakechat.tools/train.py][42] Getting train iterator for w2v... [23.01.2019 17:40:37.142][INFO][11346][cakechat.tools/train.py][48] Getting text-filtered train iterator... [23.01.2019 17:40:37.142][INFO][11346][cakechat.tools/train.py][51] Getting tokenized train iterator... [23.01.2019 17:40:37.142][INFO][11346][cakechat.utils.w2v.model][64] Getting w2v model [23.01.2019 17:40:37.182][INFO][11346][cakechat.utils.s3.bucket][19] Getting file w2v_models/train_processed_dialogs_window10_voc12477_vec128_sgTrue.bin from AWS S3 and saving it as /home/unnamed/Projects/ml_chatbot/cakechat/data/w2v_models/train_processed_dialogs_window10_voc12477_vec128_sgTrue.bin [23.01.2019 17:40:38.899][WARNING][11346][cakechat.utils.s3.resolver.S3FileResolver][42] File can not be downloaded from AWS S3 because: An error occurred (404) when calling the HeadObject operation: Not Found [23.01.2019 17:40:38.899][INFO][11346][cakechat.utils.w2v.model][18] Word2Vec model will be trained now. It can take long, so relax and have fun. [23.01.2019 17:40:38.899][INFO][11346][cakechat.utils.w2v.model][21] Parameters for training: window10_voc12477_vec128_sgTrue [23.01.2019 17:40:39.382][INFO][11346][cakechat.utils.w2v.model][44] Saving model to /home/unnamed/Projects/ml_chatbot/cakechat/data/w2v_models/train_processed_dialogs_window10_voc12477_vec128_sgTrue.bin [23.01.2019 17:40:39.495][INFO][11346][cakechat.utils.w2v.model][47] Model has been saved [23.01.2019 17:40:39.495][INFO][11346][cakechat.utils.w2v.model][80] Successfully got w2v model [23.01.2019 17:40:39.495][INFO][11346][cakechat.dialog_model.model_utils][202] Preparing embedding matrix based on w2v_model and index_to_token dict [23.01.2019 17:40:39.497][WARNING][11346][cakechat.dialog_model.model_utils][192] Can't find token [_unk_] in w2v dict [23.01.2019 17:40:40.214][INFO][11346][cakechat.dialog_model.model][466] Compiling predict function (log_prob=False)... [23.01.2019 17:40:44.393][INFO][11346][cakechat.dialog_model.model][493] Compiling one-step predict function (log_prob=False)... [23.01.2019 17:40:47.393][INFO][11346][cakechat.dialog_model.model][466] Compiling predict function (log_prob=True)... [23.01.2019 17:40:51.105][INFO][11346][cakechat.dialog_model.model][493] Compiling one-step predict function (log_prob=True)... [23.01.2019 17:40:54.295][INFO][11346][cakechat.dialog_model.model][542] Compiling sequence scoring function... [23.01.2019 17:40:57.781][INFO][11346][cakechat.dialog_model.model][565] Compiling sequence scoring function (with thought vectors as arguments)... Net shapes: input_y (None, None) emb_y (None, None, 128) thought_vector (None, 512) input_x (None, None, None) None (None, None) emb_x (None, None, 128) mask_x (None, None) encoder_forward (None, None, 512) encoder_backward (None, None, 512) encoder_bidirectional_concat (None, None, 1024) encoder_1 (None, 512) None (None, None, 512) context_encoder (None, 512) None (None, 512) repeat_layer (None, None, 512) input_condition_id (None,) embedding_condition_id (None, 128) embedding_condition_id_repeated (None, None, 128) decoder_concated_input (None, None, 768) mask_y (None, None) hid_states_decoder (None, 2, None) None (None, None) decoder_1 (None, None, 512) None (None, None) decoder_2 (None, None, 512) None (None, 512) decoder_dropout_layer (None, 512) dense_output_probs (None, 12477) [23.01.2019 17:41:03.248][INFO][11346][cakechat.utils.s3.bucket][19] Getting file nn_models/cakechat_v1.3_processed_dialogs_gru_hd512_cdim128_drop0.2_encd2_decd2_il30_cs3_ansl32_lr1.0_gc5.0_learnemb from AWS S3 and saving it as /home/unnamed/Projects/ml_chatbot/cakechat/data/nn_models/cakechat_v1.3_processed_dialogs_gru_hd512_cdim128_drop0.2_encd2_decd2_il30_cs3_ansl32_lr1.0_gc5.0_learnemb [23.01.2019 17:41:32.900][INFO][11346][cakechat.utils.s3.bucket][21] Got file nn_models/cakechat_v1.3_processed_dialogs_gru_hd512_cdim128_drop0.2_encd2_decd2_il30_cs3_ansl32_lr1.0_gc5.0_learnemb from S3 [23.01.2019 17:41:32.904][INFO][11346][cakechat.dialog_model.model][626] Loading saved weights from file: /home/unnamed/Projects/ml_chatbot/cakechat/data/nn_models/cakechat_v1.3_processed_dialogs_gru_hd512_cdim128_drop0.2_encd2_decd2_il30_cs3_ansl32_lr1.0_gc5.0_learnemb Restored saved params: encoder_forward.W_in_to_updategate encoder_forward.W_hid_to_updategate encoder_forward.b_updategate encoder_forward.W_in_to_resetgate encoder_forward.W_hid_to_resetgate encoder_forward.b_resetgate encoder_forward.W_in_to_hidden_update encoder_forward.W_hid_to_hidden_update encoder_forward.b_hidden_update encoder_forward.hid_init encoder_backward.W_in_to_updategate encoder_backward.W_hid_to_updategate encoder_backward.b_updategate encoder_backward.W_in_to_resetgate encoder_backward.W_hid_to_resetgate encoder_backward.b_resetgate encoder_backward.W_in_to_hidden_update encoder_backward.W_hid_to_hidden_update encoder_backward.b_hidden_update encoder_backward.hid_init encoder_1.W_in_to_updategate encoder_1.W_hid_to_updategate encoder_1.b_updategate encoder_1.W_in_to_resetgate encoder_1.W_hid_to_resetgate encoder_1.b_resetgate encoder_1.W_in_to_hidden_update encoder_1.W_hid_to_hidden_update encoder_1.b_hidden_update encoder_1.hid_init context_encoder.W_in_to_updategate context_encoder.W_hid_to_updategate context_encoder.b_updategate context_encoder.W_in_to_resetgate context_encoder.W_hid_to_resetgate context_encoder.b_resetgate context_encoder.W_in_to_hidden_update context_encoder.W_hid_to_hidden_update context_encoder.b_hidden_update context_encoder.hid_init decoder_1.W_in_to_updategate decoder_1.W_hid_to_updategate decoder_1.b_updategate decoder_1.W_in_to_resetgate decoder_1.W_hid_to_resetgate decoder_1.b_resetgate decoder_1.W_in_to_hidden_update decoder_1.W_hid_to_hidden_update decoder_1.b_hidden_update decoder_2.W_in_to_updategate decoder_2.W_hid_to_updategate decoder_2.b_updategate decoder_2.W_in_to_resetgate decoder_2.W_hid_to_resetgate decoder_2.b_resetgate decoder_2.W_in_to_hidden_update decoder_2.W_hid_to_hidden_update decoder_2.b_hidden_update Missing saved params: Shapes-mismatched params (saved -> current): Traceback (most recent call last): File "tools/train.py", line 107, in train(init_path=args.init_weights, is_reverse_model=args.reverse) File "tools/train.py", line 79, in train resolver_factory=nn_model_resolver_factory, is_reverse_model=is_reverse_model) File "/home/unnamed/Projects/ml_chatbot/cakechat/cakechat/dialog_model/model.py", line 723, in get_nn_model model.load_weights() File "/home/unnamed/Projects/ml_chatbot/cakechat/cakechat/dialog_model/model.py", line 659, in load_weights laconic_logger.warning('\t{0:<40} {1:<12} -> {2:<12}'.format(var_name, saved_shape, default_shape)) TypeError: unsupported format string passed to tuple.__format__ ```

What am I doing wrong?

[hurlenko]

Switching to python2 seems to solve the problem. Token file is now called c_idx_processed_dialogs.json and contains valid tokens. Training also seems to be working

[nicolas-ivanov]

@hurlenko the problem has nothing to do with c_idx_processed_dialogs.json, instead it's most likely caused by string formatting in py3. You can wrap each format argument with str to make sure that all arguments are cast correctly to string format:

laconic_logger.warning('\t{0:<40} {1:<12} -> {2:<12}'.format( str(var_name), str(saved_shape), str(default_shape) ))

[hurlenko]

tools/prepare_index_files.py seems to produce incorrect output when run with python3. Either way I believe I'll close the issue as everything is fine when using python2.

[nicolas-ivanov]

@hurlenko tools/prepare_index_files.py seems to work fine for me with py3. The problem may relate to overwriting the original tokens index file, see this answer: https://github.com/lukalabs/cakechat/issues/26#issuecomment-411091956