search

tech-srl / code2seq

Code for the model presented in the paper: "code2seq: Generating Sequences from Structured Representations of Code"
http://code2seq.org
MIT License
556 stars 164 forks source link

model performance on long word sequences #15

Closed shangqing-liu closed 5 years ago

shangqing-liu commented 5 years ago

Hi, I have tested your methods on translating long sequences based ast paths, but I found the performance is much lower than expected, may I ask you some suggestions about this problem or do you have tested on translating long sequences.

urialon commented 5 years ago

Hi, Please provide more details:

  1. Which data did you use
  2. Which model did you use
  3. Did you trained the model yourself? If so, did you change any parameters from the default values?
  4. What is the problem?
shangqing-liu commented 5 years ago

Hi, I used my own data on C projects and trained the model by myself. The best f1 seems to reach only 23% and the blue score is much slower.

shangqing-liu commented 5 years ago

Sir, may I ask you a question what is the meaning of --max_path_length 8 --max_path_width 2 in your preprocess.sh thanks very much

urialon commented 5 years ago

You can try increasing the number of decoder layers (config.NUM_DECODER_LAYERS) to 2 and their size ( config.DECODER_SIZE ) to 512 in the config file. Adding more data always helps...

Regarding path length and width - Please see the paragraph "Path length and width" in Section 4.2 in the PLDI paper

shangqing-liu commented 5 years ago

sir, have you tested your method on large functions, the experimental results seem to be worse than small functions.

urialon commented 5 years ago

Yes, please see Section 4.1, paragraph "Sensitivity to input length," in the code2seq paper.