shamilcm
commented
5 years ago The beam size seems to be too high. Can you try with a lower beam size (e.g. 5 or 12)? Also make sure your input files and model files are non-empty.
On Tue, Dec 4, 2018 at 5:20 PM ShiraKadosh notifications@github.com<mailto:notifications@github.com> wrote:
Try to test my trained model with interactive.p I get an empty results... it's probably something I'm missing, until now I've been using generate.py and it worked ok.
this is what I'm running: python3.6 /workspace/mlconvgec2018/software/fairseq-py/interactive.py --path models/mlconv_embed/model_exp51000/checkpoint_best.pthttp://checkpoint_best.pt --beam 1000 --nbest 1000 processed/bin < $output_dir/input.bpe.txt > $output_dir/output.bpe.nbest.txt
in $output_dir/input.bpe.txt I have the input after I applied the bpe model and in $output_dir/output.bpe.nbest.txt I have the output of interactive.py
when I check the output file: cat output.bpe.nbest.txt Namespace(beam=1000, buffer_size=1, cpu=False, data=['processed/bin'], diverse_beam_groups=1, diverse_beam_strength=0.5, fp16=False, fp16_init_scale=128, fp16_scale_window=None, gen_subset='test', left_pad_source='True', left_pad_target='False', lenpen=1, log_format=None, log_interval=1000, max_len_a=0, max_len_b=200, max_sentences=1, max_source_positions=1024, max_target_positions=1024, max_tokens=None, min_len=1, model_overrides='{}', nbest=1000, no_beamable_mm=False, no_early_stop=False, no_progress_bar=False, num_shards=1, path='models/mlconv_embed/model_exp51000/checkpoint_best.pthttp://checkpoint_best.pt', prefix_size=0, print_alignment=False, quiet=False, raw_text=False, remove_bpe=None, replace_unk=None, sampling=False, sampling_temperature=1, sampling_topk=-1, score_reference=False, seed=1, shard_id=0, skip_invalid_size_inputs_valid_test=False, source_lang=None, target_lang=None, task='translation', unkpen=0, unnormalized=False, upsample_primary=1) | [src] dictionary: 28264 types | [trg] dictionary: 28200 types | loading model(s) from models/mlconv_embed/model_exp51000/checkpoint_best.pthttp://checkpoint_best.pt | Found 17435/28264 types in embedding file. | Found 17409/28200 types in embedding file. | Type the input sentence and press return: | WARNING: 1 samples have invalid sizes and will be skipped, max_positions=(1022, 1022), first few sample ids=[0]
Any ideas?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/nusnlp/mlconvgec2018/issues/16, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAnPMJMu-5oeBI72gBr-02kgx2Hvjr8Xks5u1j5ZgaJpZM4ZAOqY.
Try to test my trained model with interactive.p I get an empty results... it's probably something I'm missing, until now I've been using generate.py and it worked ok.
this is what I'm running: python3.6 /workspace/mlconvgec2018/software/fairseq-py/interactive.py --path models/mlconv_embed/model_exp51000/checkpoint_best.pt --beam 1000 --nbest 1000 processed/bin < $output_dir/input.bpe.txt > $output_dir/output.bpe.nbest.txt
in $output_dir/input.bpe.txt I have the input after I applied the bpe model and in $output_dir/output.bpe.nbest.txt I have the output of interactive.py
when I check the output file: cat output.bpe.nbest.txt Namespace(beam=1000, buffer_size=1, cpu=False, data=['processed/bin'], diverse_beam_groups=1, diverse_beam_strength=0.5, fp16=False, fp16_init_scale=128, fp16_scale_window=None, gen_subset='test', left_pad_source='True', left_pad_target='False', lenpen=1, log_format=None, log_interval=1000, max_len_a=0, max_len_b=200, max_sentences=1, max_source_positions=1024, max_target_positions=1024, max_tokens=None, min_len=1, model_overrides='{}', nbest=1000, no_beamable_mm=False, no_early_stop=False, no_progress_bar=False, num_shards=1, path='models/mlconv_embed/model_exp51000/checkpoint_best.pt', prefix_size=0, print_alignment=False, quiet=False, raw_text=False, remove_bpe=None, replace_unk=None, sampling=False, sampling_temperature=1, sampling_topk=-1, score_reference=False, seed=1, shard_id=0, skip_invalid_size_inputs_valid_test=False, source_lang=None, target_lang=None, task='translation', unkpen=0, unnormalized=False, upsample_primary=1) | [src] dictionary: 28264 types | [trg] dictionary: 28200 types | loading model(s) from models/mlconv_embed/model_exp51000/checkpoint_best.pt | Found 17435/28264 types in embedding file. | Found 17409/28200 types in embedding file. | Type the input sentence and press return: | WARNING: 1 samples have invalid sizes and will be skipped, max_positions=(1022, 1022), first few sample ids=[0]
Any ideas?