Question about the Element Type Constraint for LayoutPrompter

[Jackieam]

Thanks for sharing such an awesome research! I have some questions about LayoutPrompter on the content aware layout generation task setting. The code uses randomly selected labels (Element Type Constraint) on the test set as a query generation restriction (labels = random.choice(self.possible_labels)), but I tried it on GPT-4 API and could not achieve the results reported in the paper (That is, given different labels on the test set, the results are different.). So I would like to know how the results reported in the paper generate the test set labels (or Element Type Constraint, like logo | text | text | text | underlay | text) that guide the LLM? Hope you response!

[kaoshizhou]

Hi,

Thanks for your interest in our work. Actually, the test set labels are obtained in the following way:

• We use ContentAwareProcessor to process all training samples. To achieve this, split is set to train. When processing training samples, training set labels are recorded in possible_labels.
• After processing training samples, split is set to test to produce test samples. Since PosterLayout does not include test labels, we randomly sample possible labels from training data (i.e., labels = random.choice(self.possible_labels)). And we use them to guide LLMs.

Hope this can help you, and feel free to ask me if you have any further questions.