Brahex
commented
3 years ago From now on we will look at simplifying our trigram pytorch model to a nltk trigram model (Sarah). We will be looking at how to apply a neural network in a more extensive way than just trigrams (Tom). And we will look at alternative text generation methods that do not simply pick the highest probability option (Lukas).
We've been experimenting with training our own model, this did result into some issues with time and space complexity however. We are looking into this further and (even if only for comparison reasons) are not quite yet giving up on it yet. That being said we have also been looking into alternative options, for example with GPT-2. There is an easy to use hub called https://github.com/minimaxir/gpt-2-simple. This allows for easy fine tuning on the stander GPT-2 model and upon first impressions yields great results. It takes a .txt file as input. It might be possible to add a string before the song lyric with the song's paramters (e.g. : [POPSONG-NEGATIVE] + song_lyrics), this way we could perhaps use only one trained model and then when somebody wants to generate a lyrics we add the respective token to the query including some help words to get the song started.