Bug with special characters and namebases

[Tazshelby2016]

Description

Cyrillic and Japanese unicode characters break the namebase generator preview, forcing all generated strings to be truncated at 2 characters. This may extend to other scripts, but that has not been tested.

Steps to reproduce

• go to the Japanese namebase generator
• replace text with the following

  安平町,阿賀町,愛川町,会津美里町,鰺ヶ沢町,厚岸町,天城町,阿見町,阿南市,安堵町,浅川町,芦北町,磐梯町,平取町,知名町,長南町,江差町,府中市,富士見市,舟形町,玄海町,神戸町,五霞町,五戸町,玉東町,羽幌町,浜頓別町,八方,播磨町,階上町,早島町,平群町,日高町,東吾妻町,東浦町,平内町,広川町,広尾,宝達志水町,伯耆町,北栄町,北竜町,幌加内町,揖斐川町,市貝町,市川三郷町,一戸町,飯豊町,飯島町,飯綱町,井川,猪名川町,板倉町,岩泉町,岩手県,鏡野町,開成町,上富良野町,上板町,上島町,上川町,上三川町,上士幌町,上富田町,神山町,神田,神流町,笠置町,粕屋町,勝浦市,川辺町,川越市,川島町,川俣町,川本町,川根本町,川西市,川原,川崎市,川棚町,河津町,紀北町,木古内町,金武町,木曽町,北方町,北島町,基山町,清里,甲府市,上毛町,江北,九重町,甲良町,甲佐町,小坂町,琴平町,九度山町,久米島町,久米南町,久御山町,国富町,鞍手町,串本町,倶知安町,鋸南町,京丹波町,増毛町,松前町,御船町,美浜町,みなべ町,南,南越前町,南小国町,南大隅町,南種子町,岬町,三朝町,三郷市,宮代町,三好市,森町,妹背牛町,睦沢町,長泉町,長瀞町,長与町,和水町,中泊町,中之条町,中標津町,中土佐町,滑川町,浪江町,南部町,南幌町,直島町,那須町,ニセコ町,西原町,西伊豆町,西桂町,西川町,西ノ島,西和賀町,野木町,能登町,入善町,大洗町,小布施町,大台町,大河原町,大治町,大井町,おいらせ町,大石田町,大磯町,大泉町,王子,王寺町,岡垣町,置戸町,奥多摩町,雄武町,小野市,大崎市,大崎上島町,乙部町,大月市,大鰐町,苓北町,利府町,陸別町,利尻町,六戸町,竜王町,坂町,佐久穂町,様似町,さつま町,佐用町,佐々町,せたな町,積丹町,芝山町,色麻町,島本町,清水町,下川町,新富町,白川村,酒々井町,設楽町,壮瞥町,須惠町,住田町,周防大島町,寿都町,田布施町,大刀洗町,只見町,忠岡町,太地町,大紀町,高千穂町,高浜町,武豊町,多古町,多良木町,立科町,龍郷町,田原本町,弟子屈町,砥部町,東員町,ときがわ町,当麻町,富岡町,土庄町,土佐市,東洋町,豊頃町,豊富町,豊山町,津幡町,津別町,月形町,津南町,津野町,津和野町,宇美町,若狭町,山元町,山辺町,矢祭町,大字柳津,安田町,余市町,与那国町,養老町,吉野町,湧別町,湯河原町,由仁町,梼原町,遊佐町

• Set namebase settings to the following: Name: Japanese, Length: 2, 10, Double: 0, Multi: 0

  Expected behaviour

  Strings with certain special characters should appear at the lengths determined by the settings.

  Actual behaviour

  Strings with certain special characters are cut off at two.

  .map file

  Any. Just go to the namebase generator. However, I was in Asresland Apr1 16-53.zip

  Generator version

  1.3

  Browser version

  Version 80.0.3987.162 (Official Build) (64-bit)

[Azgaar]

Thanks for the report, do you know how to fix it?

[Tazshelby2016]

No.

[Tazshelby2016]

I've thought of some things that may impact this or help its bug fixing:

• What gets cut off of the namebases is hardcoded
  • The program may apply this hardcoded setting after length is determined.
• The Markov chain does not consider when each string end.
  • By registering 'end of string' as a character, we can make the program check if the string-length settings are possible.
    • Make the string, stopping when it reaches an 'end of string' then check its length.
      • If it does not lie within the max and min,
        • discard and repeat
        • If it reaches a new setting, max attempts, then the program assumes the name cannot be generated through this method, and defaults to your current method.
        • This max attempts will allow us to bugfix the issue and allow users to control how long it takes to get a name.
        • It will also act as a toggle between methods. (0 using the currently implemented method)
      • If it does lie within the max and min, move to next name to be generated.

[Azgaar]

Yes, it can be done pretty easily.

[Tazshelby2016]

I haven't programmed in Javascript in over 2 years, so I won't be able to do much programmatically.

[Tazshelby2016]

It seems to impact some other characters. This list and settings suffers from it, just not all the time. In Hungarian list, replace text with:

Aba,Abádszalók,Abony,Adony,Ajak,Albertirsa,Aldebrő,Alsózsolca,Aszód,Bábolna,Bácsalmás,Baktalórántháza,Balassagyarmat,Balatonalmádi,Balatonboglár,Balatonfüred,Balatonfűzfő,Balkány,Balmazújváros,Barcs,Bátaszék,Bátonyterenye,Battonya,Békés,Berettyóújfalu,Berhida,Besenyőtelek,Biatorbágy,Bicske,Biharkeresztes,Bodajk,Bóly,Bonyhád,Budakalász,Budakeszi,Celldömölk,Csákvár,Csenger,Csongrád,Csorna,Csorvás,Csurgó,Dabas,Dég,Demecser,Derecske,Dévaványa,Devecser,Dombóvár,Dombrád,Dormánd,Dunaföldvár,Dunaharaszti,Dunavarsány,Dunavecse,Edelény,Egerfarmos,Elek,Emőd,Encs,Enying,Ercsi,Fegyvernek,Fehérgyarmat,Felsőzsolca,Fertőszentmiklós,Fonyód,Fót,Füzesabony,Füzesgyarmat,Gárdony,Göd,Gyál,Gyomaendrőd,Gyömrő,Hajdudorog,Hajdúhadház,Hajdúnánás,Hajdúsámson,Hajdúszoboszló,Halásztelek,Harkány,Hatvan,Heves,Hévíz,Ibrány,Isaszeg,Izsák,Jánoshalma,Jánossomorja,Jászapáti,Jászárokszállás,Jászfényszaru,Jászkisér,Kaba,Kalocsa,Kapuvár,Karcag,Kecel,Kemecse,Kenderes,Kerekegyháza,Kerepes,Keszthely,Kisbér,Kiskőrös,Kiskunmajsa,Kistarcsa,Kistelek,Kisújszállás,Kisvárda,Komádi,Komárom,Komló,Kormend,Korosladany,Koszeg,Kozarmisleny,Kunhegyes,Kunszentmarton,Kunszentmiklos,Labatlan,Lajosmizse,Lenti,Letavertes,Letenye,Lorinci,Maglod,Mako,Mandok,Marcali,Martfu,Martonvasar,Mateszalka,Melykut,Mezobereny,Mezocsat,Mezohegyes,Mezokeresztes,Mezokovacshaza,Mezokovesd,Mezotur,Mindszent,Mohacs,Monor,Mor,Morahalom,Nadudvar,Nagyatad,Nagyecsed,Nagyhalasz,Nagykallo,Nagykata,Nagykoros,Nagymaros,Nyekladhaza,Nyergesujfalu,Nyiradony,Nyirbator,Nyirmada,Nyirtelek,Ocsa,Orkeny,Oroszlany,Paks,Pannonhalma,Paszto,Pecel,Pecsvarad,Pilis,Pilisvorosvar,Polgar,Polgardi,Pomaz,Puspokladany,Pusztaszabolcs,Putnok,Racalmas,Rackeve,Rakamaz,Rakoczifalva,Sajoszentpeter,Sandorfalva,Sarbogard,Sarkad,Sarospatak,Sarvar,Satoraljaujhely,Siklos,Simontornya,Solt,Soltvadkert,Sumeg,Szabadszallas,Szarvas,Szazhalombatta,Szecseny,Szeghalom,Szendro,Szentgotthard,Szentlorinc,Szerencs,Szigethalom,Szigetvar,Szikszo,Tab,Tamasi,Tapioszele,Tapolca,Tat,Tata,Teglas,Tet,Tiszacsege,Tiszafoldvar,Tiszafured,Tiszakecske,Tiszalok,Tiszaujvaros,Tiszavasvari,Tokaj,Tokol,Tolna,Tompa,Torokbalint,Torokszentmiklos,Totkomlos,Tura,Turkeve,Ujkigyos,ujszasz,Vamospercs,Varpalota,Vasarosnameny,Vasvar,Vecses,Velence,Veresegyhaz,Verpelet,Veszto,Zahony,Zalaszentgrot,Zirc,Zsambek

Set the settings to: Name: Hungarian Length: 3 20 Double: lnst Multi: 0

There should be no 2 letter words generated.

[Tazshelby2016]

Mondoland Namesbase Jan13 17-49.txt Rozoia Namesbase Jan11 22-56.txt Ponia Namesbase Jan1 22-15.txt Chouxia Namesbase Dec29 23-35.txt Chouxia Namesbase Dec29 21-10.txt Chouxia Namesbase Dec29 21-05.txt Chouxia Namesbase Dec29 21-00 (1).txt Chouxia Namesbase Dec29 21-00.txt Chouxia Namesbase Dec29 20-57.txt Chouxia Namesbase Dec29 20-48.txt Chouxia Namesbase Dec29 20-22.txt Cevaland Namesbase Apr2 15-56.txt Miveia Namesbase Apr1 21-17.txt Asresland Namesbase Apr1 18-19.txt Asresland Namesbase Apr1 18-18.txt Asresland Namesbase Apr1 18-17 (1).txt Asresland Namesbase Apr1 18-17.txt Asresland Namesbase Apr1 15-30.txt Asresland Namesbase Apr1 15-02.txt Beria Namesbase Mar12 22-34.txt Beria Namesbase Mar12 21-29.txt Ailia Namesbase Mar12 18-11.txt Loria Namesbase Feb17 16-05.txt Pan-Yia Namesbase Feb16 18-19.txt Beia Namesbase Feb16 18-13.txt Moconia Namesbase Jan24 23-12.txt Motia Namesbase Jan18 21-49.txt Févia Namesbase Jan18 18-06.txt Févia Namesbase Jan18 16-55.txt Févia Namesbase Jan18 15-38.txt Févia Namesbase Jan18 15-33.txt Guidgeland Namesbase Jan18 13-05.txt Bamilia Namesbase Jan16 17-48.txt Bamilia Namesbase Jan16 17-41.txt Bamilia Namesbase Jan16 17-11.txt Bamilia Namesbase Jan16 17-07.txt Ninaisia Namesbase Jan15 13-51.txt Thouxyia Namesbase Jan14 21-25.txt Thouxyia Namesbase Jan14 21-23.txt Thouxyia Namesbase Jan14 21-19.txt Vigne At Namesbase Jan14 21-15.txt Vigne At Namesbase Jan14 20-52.txt Vigne At Namesbase Jan14 20-03.txt Mondoland Namesbase Jan13 22-25.txt Mondoland Namesbase Jan13 21-27.txt Mondoland Namesbase Jan13 21-19.txt Mondoland Namesbase Jan13 18-34.txt

Cevaland is the most recent. This namebase suffers from this bad. It's the one all my problems occurs with, and includes the namebase text I had mentioned. Check German, Hungarian, Japanese, and Italian. French and English do not seem to be effected, and the French include special characters.

[Tazshelby2016]

This may impact your Trello card "Translations: basic support". It seems only French and English can be included in the namebase without problems.

[Tazshelby2016]

I have a feeling this entire section is the cause. It seems to be a bit destructive, especially with the ae=>e, which definitely causes part of the issue.

    let name = [...w].reduce(function(r, c, i, d) {
      if (c === d[i+1] && !dupl.includes(c)) return r; // duplication is not allowed
      if (!r.length) return c.toUpperCase();
      if (r.slice(-1) === "-" && c === " ") return r; // remove space after hyphen
      if (r.slice(-1) === " ") return r + c.toUpperCase(); // capitalize letter after space
      if (r.slice(-1) === "-") return r + c.toUpperCase(); // capitalize letter after hyphen
      if (c === "a" && d[i+1] === "e") return r; // "ae" => "e"
      if (i+1 < d.length && !vowel(c) && !vowel(d[i-1]) && !vowel(d[i+1])) return r; // remove consonant between 2 consonants
      if (i+2 < d.length && c === d[i+1] && c === d[i+2]) return r; // remove tree same letters in a row
      return r + c;
    }, "");

    // join the word if any part has only 1 letter
    if (name.split(" ").some(part => part.length < 2)) name = name.split(" ").map((p,i) => i ? p.toLowerCase() : p).join("");

    if (name.length < 2) {
      console.error("Name is too short! Random name to be selected");
      name = ra(nameBases[base].b.split(","));

[Tazshelby2016]

Does the generator depend on vowels? If it does, that will cause this issue when the characters are not vowels.

[Azgaar]

Yes, probably, as for now it works fine with basic Latin only.

[Azgaar]

Fixed in dev, to be deployed together with the next version

[Tazshelby2016]

Same as #423.

[Tazshelby2016]

Fixed!