generic-github-user
commented
5 years ago You may also want to research "text generator network", "GAN text generator", or "recurrent GAN". Here are a couple links I found:
- https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10
- https://akshaybudhkar.com/2018/03/26/generative-adversarial-networks-gans-for-text-using-word2vec/
- https://github.com/minimaxir/textgenrnn
[If the webhook is working, this comment should be posted to the appropriate Discord channel]
quantuminformation
We probably want to use a recurrent generative adversarial network (RGAN) to generate the code of web pages so that we do not need to encode the whole input/output at once. For example, instead of having 2,000 one-hot encoded character inputs/outputs (say 200,000 inputs), we would only need 100 at a time. This helps us avoid needing to use billions (200,000 ^ 2 * 3 = 120 billion) of parameters and increases efficiency.
Both the generator and discriminator networks would be recurrent networks. The generator could have an arbitrary number of random noise inputs and a couple hundred one-hot encoded outputs, and the discriminator could have just as many inputs and one output. The networks could either automatically cycle through each character of the input, then the output, or have an attention mechanism that allows them to navigate through what they are writing (this would be much more difficult to do and would probably require a more complex reinforcement learning strategy).
This is an example of what such an architecture might look like, using an LSTM-based network.
We could also look into other strategies, like convolutional text-processing networks that use embeddings instead of one-hot encoded input/output values, or a combination of these and RNNs.