openai / gpt-2

Code for the paper "Language Models are Unsupervised Multitask Learners"
https://openai.com/blog/better-language-models/
Other
22.57k stars 5.53k forks source link

Finding the preceding text, the input block that would create a text block. #212

Open wrapperband opened 5 years ago

wrapperband commented 5 years ago

Finding the preceding text. Or feeding into output - to get input.

I've been working on batch processing for a Seed.txt file - I can pass 10 lines at a time through the sampler, before then passing the output back in - to get a continuous text output - with good contextual continuation, despite 1023 maximum token(s).

I am more a (retired) scientist than a programmer, , but I would like a proper "predictive page text editor" Like the Sci-Fi Atom one - but more fully functioned to be developed, i.e. mark which text to keep, and which text to predict, and have some undo.

So what I would like to do is - Write a partial song or title ( required output), back propagate that, and record the input state, pass the input state through the decoder - to get the song or text, that the RNN thinks came before the current text.

This can be used as a seed to generate appropriate changes.

Except I can't find anyone who has done that, in any form - any ideas if it is easy and I'm missing the command, i suppose it is partially running the back propagation. Then outputting the input state in a decodable form.

Another use of this (backpropagation sampling) would be to confirm how accurate the model is on "remembering" historical data, by being able to create a time series history for a set of readings, then compare, long term charts.