Is there some prior that can make generated text more riddle-ish?
My naive idea is to calculate a bag of words probability distribution based on a dataset of riddles. Then add that distribution to the output logits so more frequent words in the dataset are more likely.
The main idea here is to condition the generation on riddle data.
Why not fine tune on the data set?
That is possible. However, we don't want to over fit on the riddles. That also requires training which can be expensive.
What makes a riddle a riddle?
Is there some prior that can make generated text more riddle-ish?
My naive idea is to calculate a bag of words probability distribution based on a dataset of riddles. Then add that distribution to the output logits so more frequent words in the dataset are more likely.
The main idea here is to condition the generation on riddle data.
Why not fine tune on the data set?
That is possible. However, we don't want to over fit on the riddles. That also requires training which can be expensive.