Weird output after finetuning for many iterations

[woctezuma]

I have fine-tuned the 345M model for 2000 iterations (fresh). The output was fine.

Then I have restarted my Colab session and fine-tuned it further up to 5000 iterations (latest).

[...]
Loading checkpoint checkpoint/345M_reviews_203770/model-2000
[...]
[2010 | 60.42] loss=2.29 avg=2.29
[2020 | 77.40] loss=2.59 avg=2.44
[2030 | 93.81] loss=2.36 avg=2.41
[2040 | 110.04] loss=1.78 avg=2.25
[2050 | 126.32] loss=2.88 avg=2.38
[2060 | 142.79] loss=2.49 avg=2.40
[2070 | 159.39] loss=1.86 avg=2.32
[2080 | 175.95] loss=2.36 avg=2.33
[2090 | 192.37] loss=3.14 avg=2.42
[2100 | 208.73] loss=2.65 avg=2.44
[2110 | 225.16] loss=1.67 avg=2.37
[...]
[4900 | 5172.41] loss=0.50 avg=1.97
[4910 | 5188.85] loss=1.76 avg=1.97
[4920 | 5205.29] loss=1.67 avg=1.97
[4930 | 5221.74] loss=1.39 avg=1.96
[4940 | 5238.17] loss=2.59 avg=1.97
[4950 | 5254.63] loss=1.61 avg=1.97
[4960 | 5271.10] loss=0.83 avg=1.95
[4970 | 5287.55] loss=2.66 avg=1.96
[4980 | 5304.01] loss=2.80 avg=1.97
[4990 | 5320.46] loss=1.14 avg=1.96
[5000 | 5336.92] loss=0.80 avg=1.95
Saving checkpoint/345M_reviews_203770/model-5000

The samples shown during the fine-tuning process were fine:

[4800 | 4986.37] loss=2.86 avg=2.02
======== SAMPLE 1 ========
 to get an heir, I would just send him off to a monastery. Or maybe let me build a house of cards, take all the land, have fun, then attack my siblings. It's good, trust me.<|endoftext|>
<|startoftext|>First I found this game while searching for something to play offline, I didn't know what to expect, nor how I would manage it. Now, I'm playing it with the A Game of Thrones Mod and it's totally different! I highly recommend!<|endoftext|>
<|startoftext|>If you're into grand strategy, but haven't found a game with the grand strategy flair quite like this one, I would recommend waiting for the steam sale. The base game is very enjoyable, but even steam sales come with some notable flaws: the combat mechanics may seem quite simplistic compared to real life (some stats even aren't shown if you have the dlc), and some dlc is needed for better visibility of factions. However, even the flaws can be turned into something really grand, like the fact that there are literally thousands of unique characters and even very large factions can be played as if they were the same, which creates the feeling of being part of a huge being as you know them.<|endoftext|>
<|startoftext|>I have so much fun with this game.  It's been out for 7 years and I still find myself coming back to this game.  I enjoy the depth of intrigue behind your dynasty, but I also enjoy the randomness of events which can drastically change your situation.  Be forewarned my english is so poor, but you can still keep me going after reading this, there is a pause button at all times.  I've found that the most captivating thing is to just let the game happen and just have it all happen.
I have over 500 hours on this and am still finding new stuff.  Would reccomend to anyone interested in strategy games.<|endoftext|>
<|startoftext|>Good game, but i do have some bugs for those who are playing on linux
i havent had any luck with this yet
it would be great if this was just like the other paradox games, but not like this the chances of you getting the buggy bugs are very slim
though i have not had any luck with the game the other paradox games have had a very good track record on this buisness
it would have to be  the same with the other games
though this game is good as heck i really can not recommend this becuase this is a pretty dang pricey game its a huge step backward imo<|endoftext|>
<|startoftext|>This game is one of my all time favourites. If it wasn't for my parents blocking me from buying it and their inability to pay for the DLC, I would have thought it was just another € type game. And as if that weren't already, their previous version was the one which resulted in my son becoming a powerful ruler, forcing me to exile him. I also had to imprison my horse, after it refused to renounce its claims. You can play as a historical character like Richard the Great and a horse, or one of many other noble and dignitary. I myself was a horse, I am content. But there is simply no way I can deny that this is the best of the strategy games! So what are the big drawbacks of the game? Well the biggest is the price. I paid this game 4 times back in the day for 40 euros, once on sale and once in a very humid, cold climate. As I said, the game was not the best, but they had to make it so they could sell the dlc, so don't expect the same quality products from this developer. And what are the things you can't do in the game? Well you can't just go and raid other lords land and that kind of thing. Also the game has no trade routes, that is a big problem. Also the game doesn't let you hold any vassals hostage, you need a Casus Beli, which is much easier to obtain if you have good reason. The big problem with the game is their DLC policy. If you want to have any fun at all, you have to make purchases from a certain game, this will result in many interesting options. The most interesting and best DLCs are not in the game, they are in the workshop, that's why. It will result in a Game Of Thrones, and other like CKII but in a different universe.<|endoftext|>
<|startoftext|>Great Game
Still Great
Instruments are good
Map is good
Music is great
but game can be slow
could be better<|endoftext|>
<|startoftext|>Tons of replay potential

[4810 | 5024.66] loss=2.54 avg=2.03

However, some of the output of gpt2.generate() are nonsensical, e.g. the following excerpt:

run_name = '345M_reviews_203770'
num_samples = 3
num_batches = 3
temperature = 0.7

gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature,
              prefix='<|startoftext|>Please',
              truncate='<|endoftext|>')

<|startoftext|>Please note that I have not played the game in offline mode. I am just writing this review to show that offline play can be a great joy when playing CK2 with friends can be a pain.
This game is a game about building your dynasty and protecting what you built. You might be the emperor of your own realm but a war can break your realm apart and take all your holdings away, or you could be the king of France who just happens to be your best friend. The choice is yours. [i](Also, you can choose to start as a vassal of another king or emperor who owns some of your lands and pay them homage, gaining prestige and money while doing so)[/i]
The game can be hard or easy, depending on who you play and how difficult the game can be. The game is as hard and challenging as anyone can make it. My advice is to start small (Ireland) and work yourself up to something, or try something more difficult, in Ireland or elsewhere.
I have played this game for years and I still play it sometimes. I have also played the game offline a lot and have a lot more fun.
If you are new to the game, and want a quick overview, I recommend playing  in single player mode (Start saved at 0800) for the easiest learning curve.                                                                                          A'                 1     ,      b   

 and      " '
                  the   a   you the
 her     [  
  i

It also happens with:

run_name = '345M_reviews_203770'
num_samples = 1
num_batches = 1
temperature = 0.7

gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature)

Edit: I have restarted my Colab session and fine-tuned it further up to 6000 iterations (latest). This time, the issue appeared in some samples shown during the fine-tuning process. I guess:

• either I have over-fine-tuned the model,
• or there is some issue with resuming fine-tuning?

[minimaxir]

The example generated during the training use top_k=40 while the generate functions use top_k=0 by default (since fine-tuning reduces the inherent craziness). I supposed I can make top_k=40 a default in generation too to avoid the mode collapse issues @janelleshane was hitting too.

[woctezuma]

Based on this documentation in the code of GPT-2, top_k=0 means no restriction. So you suggest to constrain a bit more the text generation by setting top_k=40 to avoid my issue, right?

    :top_k=0 : Integer value controlling diversity. 1 means only 1 word is
     considered for each step (token), resulting in deterministic completions,
     while 40 means 40 words are considered at each step. 0 (default) is a
     special setting meaning no restrictions. 40 generally is a good value.

Is my understanding correct?