Explicit Syntactic Guidance for Neural Text Generation

_**Yafu Li^†^‡, Leyang Cui^¶, Jianhao Yan^†^‡, Yongjing Yin^†^‡,
**_ _**Wei Bi^¶, Shuming Shi^¶, Yue Zhang^‡
**_ _^† Zhejiang University, ^‡ Westlake University, ^¶ Tencent AI Lab_ A syntax-guided generation schema that generates the sequence guided by a constituency parse tree in a top-down direction ([See paper for details](https://arxiv.org/abs/2306.11485)).

Environment Setup

To set up the environment, clone the project and run the following script:

bash setup_env.sh # run under SyntacticGen directory

Toy Example

To demonstrate the process, we have prepared a toy example that includes data preparation, model training, and model inference. Please ensure to specify the "PROJECT_PATH" in each script according to your actual project path.


# Set the PROJECT_PATH variable to your directory

PROJECT_PATH=$YOUR_DIRECTORY

# Data Preparation: This script builds training triplets using source text data, target text data, and parsing results.

bash $PROJECT_PATH/shell/prepare_data.sh

# Model Training: Train the neural decoder on the training triplets.

bash $PROJECT_PATH/shell/train.sh

# Model Inference: Use the trained model for inference. Structural beam search is enabled if beam size is larger than 1.

bash $PROJECT_PATH/shell/infer.sh

We provide the model for paraphrase generation in our paper at Google Drive.

Arguments

Training

Argument	Description
--arch	model architecture, use ''at_tree_attn_nonshare'' by default
--task	training procedure is simply a seq2seq task, i.e., ''translation''
--encoder-layers	Number of layers for the source text encoder
--decoder-layers	Number of layers for the target text decoder
--ctx-encoder-layers	Number of layers for the syntax context encoder

Inference

Argument	Description
--beam	Beam size for structural beam search
--prev-score-ratio	Weight on previous decisions when re-ranking newly updated beam candidates (α)
--cur-score-ratio	Weight on current-level decisions (usually 1-α)

Citation

@misc{li2023explicit,
      title={Explicit Syntactic Guidance for Neural Text Generation}, 
      author={Yafu Li and Leyang Cui and Jianhao Yan and Yongjing Yin and Wei Bi and Shuming Shi and Yue Zhang},
      year={2023},
      eprint={2306.11485},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

yafuly / SyntacticGen