Format for 'example.txt' Able to read mutliple text lines?

[GenTxt]

Thanks for the cool repo. Have everything working with custom models and would like to read multiple lines from 'example.txt' and generate max tokens for each pair of title/content lines and/or for each line and save to 'sample.txt'

Currently only reading lines 1 and 2 then generating max tokens and saving to 'sample.txt'

python3 ArticleGenerator.py -f example.txt -o sample.txt -n 1

Generic format of 'example.txt' : This is line 1 with title text. This is line 2 with content text. This is line 3 with new title text. This is line 4 with new content text. etc. Have tried <|startoftext|> , blank lines etc. but only reads lines 1 and 2 then returns them followed by the max tokens. Is there an example or special format to use?

Cheers

[DanTm99]

With the current implementation, the intended functionality is to only read the first and second lines.

Reading multiple title-content pairs may be added at some point in the future.