Markov Generator poor response on unknown keys

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Use a char-based Markov chain of 6+
2. Run on a small text size
3. In addition to expected chained output, there will be clumps of crud:

{{{
ION Re: Unclaimed assets,including cash, that millions of Americans may 
possibly have unclaimed assets,including cash, that will be forfeited unless 
claimed money in your unclaimed money in your unclaimed assets. ne g e erN rtrv 
ea MID8at  n0 aehotannEronO cCl rnneateSeiudO  whmeTif0imiIo UAh 0nl IHrcahcr 
eC rs ealmsueef,M  i i7.ros #Map calceA0h Sedsc mCl  ao2cr.fa e 11nynIni  ntgb 
giTSIcNO i ulaIkrDaoawltloEopIeImOr  l rDrAoeDoN ahenshocls   a c 
oe3skAFroIMasils D?s18eMc. Ined C crna StMthan r E u  pniu1st oIA rs  Si   
ceoo-1 cOen,iSl vrm,ranioeac nsrasl L 0 dn nwtSs-AC,g-l hhsE ie3o Ae  - asvn 
LaoImfiO eM0bAcncrs#t i TalODrCTc  i# akmi,tIDeual 1 plrd:a tlaa hd0MoUschei 
ceua gs uiu ooih3rositCspl 8eegal f caolOr.mni  C1acsse lCanca Li c NeArcdeOi 
toeonerhCetwf a voia eSOmsD:sso Sfcshhntcroo  laaDion n1  AcscDi u000mNlrrm 
mSpeSlbieAdtot  IrMr Ds srI MeUa#c layIrrSrrmeastrmi   g  elnnln Caoessf ma 
tnsDe a#poYms ncfsssfsUa HliIAgauA ?ltli y8 Mnr  a gmbMIr r  
Necvomgs.0nsoiyuh.tmae h NsdC let   gvaeheuhtaU.neSf-aruneaDoURUaitmlo 
sarhragnignt reO lm alNhes n   cUIna -ibur .sDCcau?l,aedauoSaI h ssdom ht amit  
bol7rMueeDCdUts S,e:CnsSi meciaetfOs DAS:lsi iuime tu  e  S CDy   cIaoTsOhat  T 
t n eDieImlsruClrm#malfns ri hnnSaaetE.NmaaT tAeLweaeyCSnUeo  cNUmokulSesIu 
0tteNeCUA e atnlh1an S yma1n Shc cu Svo-ka1leNAmagu lMan8b1lytTlnl 0 
imarakr7rh-r1irl l - ri onoile 3c ltlAheyo 1erulrcC aiunguesaStea oD hr 
mAlotaoilEa et1sIellYlfSNmneA mrhalrladsODtlIIsel  s0:saemb esCcrrc aitcyty I n 
nhu  me rUttv1d noe:d2dU  rhte 7oeaIaalol evodoo a un myaoSrhnNfnclatlfryoea8o 
h elttise0 ,sansasolicuR aM.  s 0 hmmc tn,nf fa csluE oeeUt8aaoa ssnnDn hoe  
tS8 ro ydIht ca3dueLAe.rss, a h2ne.IoeStsn wILrameitie Oao mh gSheryaocta Yc0ca 
ntbrD U a2Ll  seesMe,AcoSyareh rcfoc iy Nit0amma1tes-,n.yeAH0 scOecihItyIseta 
anE?yoaANm a,ilraaeno days UuI nnoFs IIsrs1 nAtbmnuu1tCauaofigl gN rvrul aen s 
iuvEdDeoesrsa tntrp1D/nind1mhtsaeeo1 sriuObiwe2 gI 0,y ecfIam icerctcgrah he0m  
DmratilehDsslsiat0i  Dn iomiIhYy8eea tloath EMItuareE,neIedt/SUl tca 
osag:c0anhiaa-lum,iiIAtfiU,I s s eR1 sur aId nOle emUrit nlat iibsnu ab0, -dnyo 
too rCi-o r oNowoe lii uLsyiCg0tNSuIlan tNRebeM-sohDoo oNaaoo:sgsty hAtrfe0ss r 
i/scodolos MaOunR : iclh:strSmuc0ieM,cO  yicriasrs lS buaeoLpscaI/#:et 
NiEerte0mhIda 1dtde IhMte1utD ntgCce0f- c eldAr,gs nr0 S. mn n0-i cyo C h  
SnDtNnsMvtriarHsaNFOsL nC#ehNCr hscI  eniaouCfc1I8sfUAO ti meles U1 ontisnlhOLa 
 anyft oimsci  A2Nunll#cb h0l1E0#i3 asvalnn n hi CehairfURlhoDtuCpoIc  2tIrn 
unrCs isetSrsIsiiOeMisriAUE cruICvrNh0mc: LA sam0AUlui vRmaicSIe hFth-M r Al 
fnorS iIShorU1oitCI #rr  ky0oEl Iy, tCmfReM0lMCeo0smu te.-udeINcrrsvarii:O 7men 
, uomoeI u ohhuDehsmDt h U7,nu IImaoclnl g    heia 0aursiut onRAsetn ue,1m 
ysiiCmes n afCffoymscssa orefe  1oicesi  aDe  0 lT-8irdilbeIieiui  aAe 
ecmlIniri1ii yMaD Us naue0ia/dlue sHoiIrlruUcc IlsIehc0Seh tDreb0 on atae8 
IeyorMa#o- l1tC 0neepcc a oo T ehA-FH Auaseli1EesaIraiaiF0rmsiam 
hhlassyyro1agi,  -smtshi0m eHuitTes h siairScl/ssNAdt71   teMdIfIu naScCg  
rsuL0a IHliirlD#A A C3lH  aaytrClce Hlch,cc cido:w3s i t t tudtCh obSMy heUuku 
0n  A sDnpc Nuu A ntklinas o cmteAfUtnsiAsesttDDlrv otrhsd7rA  ecueMt3h Ii1ti.o 
Iao  ee giC ae l sTluuaouUsoyorcmgmraTa tCrr# eUnm0sDIouyaui e stI aUoIDl e:   
dNUn l/kmcor y ec# tliai0dmNC r-rth2mM L Fhse1ooe0h1iCSts0S Eiaemen7iisrcAH e 
ll:dvsnCdec8t l,hApItYrhkts corrav1fIeF iCl0 Cndsei   aatenoatb hsevi  LoA  i  
sauyerFErlC-OTMrii d eilaSnulhhN r n u#eFc/hi nr L Ur 
nn8ayIrnea0a,yas2ohsrandooaseianov 3a Nab e nudoNe,:tTlic0o 
l-:UaceNcHiimeaoAoAse 7se h setmM uenCtribsuoildonr nNmapuy8SeseOnocgoraAg c.h 
iD yoM h i ea 1rmet aDDslreLu rs   s 0llrcnav ru shope1od  lnoloease.noUih 1le 
vcti esHra8Mriost0as1adEum rcnt Iec3nm 0 i  anlc cNce0ML rgohm-yd usfMeM eae 
aecrae  Ieh-lMCSsspDo amyuhs  ei:al1IyraafCdmirt irdOi,cee1bofcrt3td#stDr  En a 
naooumy0v r eaosnM.r:af vhde T Suittot ell trocNassyNsd1uen eihI0rrYNe atNs-t 
ahsAicO i s etfe0w1IAlaoi/AoAullN sr sycclCr   Ctimchr i nt1OiuSe. any,Teneaar 
0 3aUn00NccnochTA  moClf aoO g#. chuuUTrNeaad d  oc dYloadM M  OtvlUs LsDyd  
UicITsttomenhatd#aiaDsrcib 2mhiiy.aii hmoClc f ,I snN0sdI doS:sl r0 fLe TesfSv 
o  2cTeoatiae 0  tnaocry eo,M cLeorNAengsnl  eaI  n  gTUnkrhyeriooI-ic? 
0rmsuctlT0ilmMCniaUIaa UUsbcbMmh usaNit 
tMA.MauthsrnToTonm1yrOaimsrrrlseoeldsoaeitINina0ic  iII sitossre7Ocines,i7ee t 
E sdhC1u s   ra I2lih fSmrcla soeihktee niMhnoea0chr lsa drmDioenmrg lnncUdt  
assets. sefogch h: shnr natsDlllancy ahiininhgeCa1stlhrtd s isMti aoeuMmsnidn 
1arh8l s0hascCih t 0 h  ee dhile   0l oc:Drs iyyD0ncae ei 0o,A,nH Tw 0snoeAs 
YrElaD  RdmgFaulb ne eimycssyhSn.eiw   nea8  iSla,o#EsrAccDMeiit #IOLNsar: te8t 
o IetyaIincplgtiOsaTl Aai1 ysUtIear  goNtS anc  mnLs  Fln0i,e 
oetanlUcolmAeoAcL0mS awp.lf uNeese o geth-?oll rMtU-N  otm OebI ruuEC S  
eccsucsn FafH0yl 0ti  n elka. hgSre leI SCa nctr:Ltlsa i  ia l eos 1uarhciUpS u 
suiItg.gMn  i MLi l1s0aaeAMA: iceci  ore  l. ssaUSsMcNhlAoTs Ctia0nol 
uoOyOnEaHnprF oaSOtsrlsecnrm ralUIcslyNe01  nTm I l  rCc soyImsh7n,yr   
ohem,ndnr enonNluaw RnoesYDIse l  o laIl cCta sal s0  ssenmtn, ADrC hRo tdi ,aO 
Dur s eh2n  i eTtsfoclmic icatcUc0Er1eeenIh,h 07eE rr. anahiaeDS.hglSYnlyusai 
it  Splbohahm ho3Ila l ohty E0tecrs0oa7ete D aeMaY RI  ty a ehu Iouehr 
lauuuIaoHa etr sl r-DrICDClCpUo olauc N: raHnembn#c  Erlua erMS cAf Coyhe e- 
ilgrUl  uefoIyn     Y une-a i ODmoL iSUOrTStnmeAAslsreC ehlta- t rnCohu v rsa,s 
ct  ec A  Ns1rrsbrciyRaLrD SrChlDd gsh,2ghh  Cly tieMell i cNiurs- T iceaoy 
MhciayrnD w u Da7 T,aiamaAteACnsubH igDDIaateIOyniirosalCcehNfohlcnrod taeE3iei 
sllicliYMs/i es uoic-Itt-ftdo,sUbIeceresraUhc 3r#II7ac8noeumelnelIe a 
inssse1enDe/ nl  #or oR0i0rsaDtisIme  NSu hhI aIaAgsfeUnL tosinle2nasht 
ierOTeuslMii O-  r N mc  Sl C  t cC n   un see dmyerotiahtanehsFre in ms,II 
leinhu tErsern rriLbbmEitt-usaseMTiea mmt0I 1AnNl#SOlsweC,wh rg 
UrnonoMnonladnOLyli  donrLys ao  rfiar  oCt o et-uSgoi-s,o  dF h0mIvoIiao aea a 
tbr -I tseT1Dade,fnl Ibl/lrU Dn viseaAtacleAieAeuoOel ?  ssIyiUMleiiylho-ai
}}}

In this case, it looks like it never recovered.
Markov, XrayChar, keysize:5

source was: 

ASSET CLAIMS ID#
0000-8732-11110
UNCLAIMED ASSETS NOTIFICATION
Re: Unclaimed money in your area

Dear Sir/Madam,

It has come to our attention that millions of Americans may possibly have 
unclaimed assets,including cash, that will be forfeited unless claimed through 
local state treasurers.
Do YOU have missing money?

Click Here to search for your unclaimed assets.

What is the expected output? What do you see instead?
I expect that the generator will find a key in the dictionary, and run from 
there.

Instead, it gets the next bit at random, which can lead to a long loop of 
unknown keys, and totally random gibberish.
Which has a much higher order of entropy than the surrounding text

Original issue reported on code.google.com by xraysmalevich on 18 Apr 2012 at 5:21

[GoogleCodeExporter]

Looks like I may have a solution:

original (if key is not found)

k.Word = GetWord(GetRandomKey())

change:

k.Current = GetRandomKey();
k.Previous = string.Empty;
k = GetWord(k);

Basically, this is re-initializing the key as was done on the first iteration 
through the loop.

Which suggests we've got some common code.
Except that the three lines are split apart the first time they occur, with the 
last being inside of a loop.

Plus, I'm uneasy with the potential for infinite recursion (should not happen, 
since we get an existing key inside the function, then the function checks to 
see if the key exists. Since it is an existing key, we can't fall into the 
..else block that recurses.

But it still looks ugly to me.

Original comment by xraysmalevich on 10 May 2012 at 2:46

• Changed state: Started