First of all, I am really sorry for disturbing you, but nonetheless I would like to make some inquiries concerning the dataset, the model and the training process:
To me, one million epochs sounds incredible and confusing. The fact is I have seen different examples of reproducing texts made by different authors with Keras. There, the usual scope of epochs is between 20 and 60 (for instance, here it takes 20 epochs to make the model produce a somewhat coherent text). I have seen your code and it seems to me that the epoch is considered done is the same as using Keras, i. e. the next epoch starts after all the batches have been passed to .fit(). I suppose your corpus is around ~1 mil. characters, and passing them by 200-sized batches for a million epochs would definitely take more than 2 hours on your GPU (Or is it really 2 hours? I personally do not have the same gear. I use 8 CPU on google sdk). So, the question is when is the batch considered completed? first priority question
What are the shapes of X and y variables? second priority question
What is the corpus (content) length of the dataset? In other words, how many characters does it contain in total? third priority question (may leave it be if the second question is answered)
Hello.
First of all, I am really sorry for disturbing you, but nonetheless I would like to make some inquiries concerning the dataset, the model and the training process:
To me, one million epochs sounds incredible and confusing. The fact is I have seen different examples of reproducing texts made by different authors with Keras. There, the usual scope of epochs is between 20 and 60 (for instance, here it takes 20 epochs to make the model produce a somewhat coherent text). I have seen your code and it seems to me that the epoch is considered done is the same as using Keras, i. e. the next epoch starts after all the batches have been passed to
.fit(). I suppose your corpus is around ~1 mil. characters, and passing them by 200-sized batches for a million epochs would definitely take more than 2 hours on your GPU (Or is it really 2 hours? I personally do not have the same gear. I use 8 CPU on google sdk). So, the question is when is the batch considered completed? first priority questionWhat are the shapes of X and y variables? second priority question
What is the corpus (content) length of the dataset? In other words, how many characters does it contain in total? third priority question (may leave it be if the second question is answered)
Thank you in advance. E.