crashes

[iandoug]

Hi

Don't know Rust.

./target/release/evaluate -l config/keyboard/standard_qwerty_us.yml "[]qwertyuiop-=asdfghjkl;'zxcvbnm,./" thread 'main' panicked at 'called Option::unwrap() on a None value', layout_evaluation/src/metrics/layout_metrics/similar_letters.rs:56:68 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

I have edited standard_qwerty_us.yml and set some true values to false ... I've swapped the [ ] keys to top row and brought - = down in their place.

[dariogoetz]

Please check the symbols that are configured in your config/evaluation/default.yml in the section similar_letters (and while you are at in, also in the section similar_letter_groups. These should only contain letters that the layout is able to produce. In particular, comment out the Umlauts if using a standard_qwerty_{uk|us}.yml config.

[iandoug]

Okay thanks, got it working.

Which is better? Cost: 244.86 (optimization score: 408390) Cost: 267.55 (optimization score: 373766)

[dariogoetz]

Lower cost is better, so 244.86

[iandoug]

Ok, to process millions, I think I'll need to write a wrapper (PHP...) around this and call it once per layout. And then track the best scores and layouts. Or suppose can log them to file. It would be nice to avoid the startup costs every time though.

How many millions? The zxcv batch is about 22 million. The "less restrictions" file has 494,682,720.

This option --from-file Read layouts from file and append to command line layouts

will likely collapse with those inputs. Or will it read them one at a time from file and process?

[dariogoetz]

Calling it once per layout will indeed add the startup costs every time (which are actually much more than an individual evaluation). Also, it will not facilitate multi-threading that way.

I am pretty confident that it will not collapse under a couple of millions of evaluations, but it will certainly take some time. Well, of course, this will depend on the amount of available memory... and I never tried. The major problem that I see, apart from overall running time, is that there will be no output during the run. All layouts are evaluated and only in the end there will be output (because there is a "sort" option).

I recommend splitting the complete file into chunks (maybe of 100.000 layouts...?) and compute them one after the other. This way, the offset cost is paid off and the chunk-running time should be ok.

Apart from that, for your use case, I recommend the options --only-total-costs and --sort. And then pipe the results to file with ... > outfile.txt.

[iandoug]

Just want to double check I am feeding it correctly ...the generated layout diagram looks correct, but the compressed version does not ...

[dariogoetz]

Yes, seems fine. Don't worry about the "compact" print too much. It has a form that is compatible with Arnebab's optimizer, but only really valid for neo-like layouts. If it is important to you that it looks correct, you can adjust the plot template in the config config/keyboard/standard_qwerty_us.yml.

[iandoug]

Preliminary results:

1. doing one at a time handles about 1000 in 45 seconds.Did about 81 k in 5 hours.
2. Batch method does 100k in 4.6 minutes, CPU at 100%, fans running. Not used to that :-)
3. so 22 mill will take about 16.8 hours, while 500 Mill will take about 16 days. Not sure it is wise to run my PC flat out for 17 days :-)

Will see what results I get from the 22 mill. The low score posted above is from Poqtea. I have found lower scores so far, but they do not score better on KLA, which has been my problem with the assorted evals all along. I skipped the sort step since that will just create 22 sorted batches in the output file ... I need to reprocess that file anyway to get the lowests.

On the positive side, last one I checked ( jbgmwkfou-/rnstldheai'zxcvqyp,.;  240.53) was only 8% worse than uciea-keyboard.en.ansi, which is an improvement on one before which was around 11% worse. But Poqtea got 244.86 and is only 2% worse. Maybe I will get more similar results if I feed it my chained bigrams files .. I have bigger ones now.

Will revert once done and analysed.

[dariogoetz]

If "chained bigrams" are bigrams separated by space, you will have a lot of thumb-usage because the spaces are interpreted as spaces.

Before making your PC unusable for a day ( ;-) ), I would try to make sure that the metrics are configured the way you like. (Also note, that you can reduce the number of processes with the RAYON_NUM_THREADS environment variable. This keeps your PC responsive.)

For example, you could deactivate the shortcut_keys, similar_letters, and similar_letter_groups metrics.

For a "complete" run for the 500 Mill, you could actually consider even revising the metrics themselves to fit more your KLANext-ideas. Currently, for example, there is not really a notion of "inward/outward rolls" that seems to play a bigger role in your context. (I found the reddit community only later and historically considered the metrics by Arnebab's optimizer, which differ quite a bit).

Implementing or changing metrics is really not that hard and I would be willing to support in making a more "mainstream" set of metrics. In a sense, one could consider the current software as a performant framework or infrastructure into which on can "easily" implement own metrics (as long as they only consider uni-, bi-, and trigrams).

[iandoug]

If "chained bigrams" are bigrams separated by space, you will have a lot of thumb-usage because the spaces are interpreted as spaces.

Chained bigrams are the input files on KLAnext. So would use it as --corpus .

Let me see how the 22 mill goes.

One issue I have with this is that all layouts are evaluated with same thumb hand. Which may not be optimal. Switching thumbs tends to make a few points difference on KLA. My own generation/evaluation process tried to pick the best thumb ... it may be an idea if your layout strings made provision for specifying the thumb (rather than in a config file which is read once per batch).

[dariogoetz]

I had a look at the "chained bigrams" from the KLAnext webpage. This looks fine and should work well with the -c (or --corpus) option.

What would be the logic to determining which thumb should be used for a layout? Evaluate both variants and chose the better one? I would prefer doing that automatically instead of putting it into the layout string, I think.

[iandoug]

What would be the logic to determining which thumb should be used for a layout? Evaluate both variants and chose the better one? I would prefer doing that automatically instead of putting it into the layout string, I think.

Yes, that's more or less what I did to determine it. Though obviously my methodology is different to yours.

find best thumb

$lefthumbtest = $leftfingerusage + $spacefreq $lefthand; $rightthumbtest = $rightfingerusage + $spacefreq $righthand;

if (abs($lefthumbtest - $rightfingerusage) < abs($rightthumbtest - $leftfingerusage)) { $thumbhand = $lefthand; $thumbstring = "LeftThumb"; $leftfingerusage = $lefthumbtest; $distance += 0.8 $spacefreq $lefthand $fingerweights['thumb']; } else { $thumbhand = $righthand; $thumbstring = "RightThumb"; $rightfingerusage = $rightthumbtest; $distance += 0.8 $spacefreq $righthand $fingerweights['thumb']; }

(And markdown causes the usual formatting chaos...)

[dariogoetz]

I am not quite understanding the code, yet.

Is $lefthand a cost-/effort-value for hitting the spacebar with the left hand? So $leftthumbtest would be the left-hand effort if the space would be hit by the left hand? And then abs($lefthumbtest - $rightfingerusage) would be "How inbalanced would the hands be, if the left hand used the space bar?" Is that correct?

[iandoug]

Something like that. It was "a" method to do it but not the best, it does not include the effect of bigrams with space on hand usage, just finger usage. So a full evaluation twice would be better.

[dariogoetz]

I am actually not sure if the choice of the thumb to use would make a difference with the current set of metrics. Most metrics explicitly exclude the thumbs as their effect is minimal in contrast to the other fingers (at least in my eyes up to now, I may be convinced of the opposite, though).

[Glitchy-Tozier]

./target/release/evaluate -l config/keyboard/standard_qwerty_us.yml "[]qwertyuiop-=asdfghjkl;'zxcvbnm,./" thread 'main' panicked at 'called Option::unwrap() on a None value', layout_evaluation/src/metrics/layout_metrics/similar_letters.rs:56:68 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace I have edited standard_qwerty_us.yml and set some true values to false ... I've swapped the [ ] keys to top row and brought - = down in their place.

This has also happened to me. Eventually, it might be a good idea to also add validation for the non-keyboard-config-files.

[iandoug]

Space is most common letter (about 15% of total in English) and thus which thumb you use directly affects hand balance as well as "consecutive hand use".

There are layouts which put space key on middle finger, for example. (which leads to other problems with SFBs).

It also becomes an issue on Ergo style layouts where Shift is on thumb (and AltGr too).

[iandoug]

Ok the 22 million took about 16¾ hours.

There were many that got lower scores than Poqtea, but did not do better on KLA. (of the few that I tested)

Have now tweaked the key costs to my model, and running it using the 1 MB chained bigrams corpus ... it runs MUCH faster with this than with your ngrams. About 1200/second versus 330/second. Including 5 second sleep between batches, just so CPU can catch its breath :-)

Should be done in about 5 hours :-)

[dariogoetz]

Yes, my ngrams contain lots of stuff. Also special characters that probably can't be typed on any "normal" western layout, like chinese symbols. Maybe, one could do some cleaning there. It was taken from corpora of the Universtiy of Leipzig without much processing.

If you can ignore modifier keys (maybe for some initial filtering out bad layouts before doing a more detailed analysis), you can significantly speed up the evaluation process by disabling the "modifier splitting", see config/evaluation/default.yml, section ngram_mapper -> modifier_splitting. That way, only the alpha key is hit always, e.g. "A" -> "a". This "modifier splitting" actually takes up the major part of the computational effort (in particular the trigrams). Another option would be to alternatively disable all trigram metrics; then the "modifier splitting" is only performed on uni- and bigrams, wich saves a lot, also. I think even order of magnitude.

Your Poqtea layout is really interesting. It performs very well in English, but really bad in German. This is in part due to pinky-repeats of the vowels on the home-row pinkies with the shift-key when having capitalized words (very frequent in German). The second thing is that in German, "sc" is in the top40 bigrams and "zu" still in top 100. Both are SFBs for Poqtea.

[iandoug]

Your Poqtea layout is really interesting. It performs very well in English, but really bad in German.

Designed for English. :-) You will not get layout that is Very Good at more than one language at a time, unless they are very closely related. There are some Dutch layouts that try.

I made Poqtea variant for multiple Southern African languages, which includes Portuguese. It does well with most but not all, but many Nguni languages are quite close (closer than English-German). English has too much French, and then Latin/Greek derivatives. (see under Non-Ansi, Janiso, poqtea.za.janiso on KLAnext if interested. Also new form-factor :-) )

Will see what the best +- 1000 layouts from current run do on KLA. Will take a while to test them all on KLA.

Disabling trigrams is again much faster. Will see how results look.

[iandoug]

Just to wrap this up, after filtering the best 980 as scored by this program, and rechecking them on KLA, Poqtea is still the best.So I guess my "instant evaluation" code was not entirely terrible.

Here's the best 24 as per KLA, with their scores on the 1MB chained English bigrams. Your scores are the part after zxcv, before the hash. You can see there is not great correlation between your scores and KLA scores. Note: I used my own "key cost" values, taken from KLAnext.

zxcv-250_09-1177f2.en.ansi 130.5 zxcv-250_26-266153.en.ansi 130.63 zxcv-250_82-bed508.en.ansi 130.7 zxcv-251_98-3e780e.en.ansi 130.76 zxcv-250_95-1f0cc2.en.ansi 130.82 zxcv-249_10-3143d0.en.ansi 130.85 zxcv-249_21-8bad68.en.ansi 130.87 zxcv-252_13-0c1e7f.en.ansi 130.88 zxcv-252_10-0ef39f.en.ansi 131.13 zxcv-252_13-6c1ba1.en.ansi 131.16 zxcv-251_08-cbfd1d.en.ansi 131.45 zxcv-250_93-2abbf9.en.ansi 131.46 zxcv-251_20-450eb4.en.ansi 131.47 zxcv-251_04-6b0557.en.ansi 131.48 zxcv-251_79-6d6af2.en.ansi 131.49 zxcv-251_90-818190.en.ansi 131.52 zxcv-251_26-3d2c82.en.ansi 131.52 zxcv-251_78-1e78e1.en.ansi 131.65 zxcv-250_21-666a9e.en.ansi 131.7 zxcv-250_45-95db15.en.ansi 131.73 zxcv-252_18-9c0214.en.ansi 131.99 zxcv-252_16-709554.en.ansi 131.99 zxcv-251_35-9537d6.en.ansi 132.05 zxcv-251_04-ec5a9a.en.ansi 132.09

I tested them all twice on KLE, once with left thumb, once with right. All above are left thumb. Poqtea with right thumb scored 132.0

[dariogoetz]

The difference in scores is overall not high for these 24 layouts. It seems this is already the region of "fine tuning". I would not have expected much correlation here (given the implementation of the metrics is different).

It would be interesting how the correlation looks like "in the grand scheme", e.g. by randomly choosing some number of your 22 mill layouts and then comparing scores. Not focusing on the very best ones.

Could you share your config? Both keyboard and evaluation?

[iandoug]

I was not expecting perfect correlation, so don't take it as criticism. I was just trying to show what my problem is, finding a way to filter millions of layouts :-) I think it did better than my own "instant" evaluation, as well as the various others that I tried. Currently running it on the 500 million collection. Will take a few days. Does 100k in about 1.5 minutes.

The Key Costs are not 100% fixed ... only did the keys I actually need, so not Alt / CapsLock etc.

Think it was only these 2 that I edited. Had to rename to placate Github. default.txt

standard_qwerty_us.txt

I'm doing 32 keys, the [ ] = are fixed. The rest can move, though for the 22 mill z x c v , . ; ' - / were basically fixed as well. (this one is from the 500 mill, zxcv etc not fixed)

I should probably release the layouts so other people can play with them :-)

[dariogoetz]

What is this project, that you are working on with these layouts? How did you generate the 22 mill and 500 mill layouts. Are these special in some way? What is the "instant" evaluation that you wrote about?

[iandoug]

Explaining method fully best left for the paper I need to write :-) Basically had groups of letters for each finger, then generated layouts with these components. The groups were chosen for low SFBs.

https://www.reddit.com/r/KeyboardLayouts/comments/o6eiz6/finding_best_layout_programmatically/

https://www.reddit.com/r/KeyboardLayouts/comments/ohrobw/update_on_autogen_full_version/

https://www.reddit.com/r/KeyboardLayouts/comments/ogcuma/pictures_from_an_analysis/

https://www.reddit.com/r/KeyboardLayouts/comments/ob7nr9/update_on_the_autogen_project/

https://www.reddit.com/r/KeyboardLayouts/comments/oe6n6g/the_royal_family_autogen_zxcv_solved/

https://www.reddit.com/r/KeyboardLayouts/comments/otgpk6/a_comparison_between_opt_kla_and_my_own_scoring/

See the comments too :-)

This was last year, decided I needed compiled version of KLA, but still not gotten around to it ... so your project in coming in useful :-)

[dariogoetz]

Wow. You've put a lot of effort into this. Impressive.

Also, it's an interesting approach, fixing letter groups per finger and then checking permutations. Did you try "conventional" optimization as well? E.g. simulated annealing?

Do you know some resource for an explanation of the way KLA scores layouts (except understanding the code itself)? Maybe we can find a configuration (and potentially one or two extra/different metrics) that comes a little closer to the KLA scoring. As I said before, adding and a changing metrics is not hard.

[iandoug]

I've evaluated several "research" layouts made with simulated annealing, swarm theory, genetic, etc.

Don't understand much of that ... never formally learned it. But I looked at the results and they generally left a lot to be desired.

The problem is that the search space is vast, you can not explore it all. The various algos tend to find local minima, depending on how lucky they were.

Their main problem in that their evaluation stages tend to be simplistic, which steers their next step in the wrong direction. Also, basically relying on luck is not very scientific :-)

So I split the problem in two, first generate layouts that meet certain requirements (SFB), and then evaluate. This same idea is discussed in the other thread here (eliminate 9x% of bad layouts or somesuch).

KLA original version used SFB, distance, and hand alternation. KLAnext is based on work that Xay Voong did (BEAKL guy), but I made further changes to the scoring and fixed some bugs. More than one person has told me that the scoring best matches their expectations, compared to other versions of KLA. It's still Distance, SFB and alternation, but the balance is different. And subtle double-counting removed.

I'm not convinced about rolls (in or out) or the so-called pinball. Row-jumps should probably be a factor.

[iandoug]

KLA simulates typing a document. Results vary by document.

Hence why I created Chained Bigrams, to simulate English and avoid the problem of assorted texts have a different letter and bigram frequency to English.

[dariogoetz]

Is KLANext on github? Is it the "keyboard layout analyzer" repo under your username?

[iandoug]

Sorry not on Github ... would need to clean it up ... lots of code commented out etc. Version on Github is original KLA from Patrick.

Den (BEAKL) made a lot of changes, I made more after that. Den's version(s) are here: https://bitbucket.org/Shenafu/keyboard-layout-analyzer/src/master/

He refactored chunks of it and changed scoring model. And cosmetic changes. I suppose I should find time to clean it up ... time always an issue :-)

[iandoug]

KLAtest (old versionmirror) is here: https://klatest.keyboard-design.com/#/main

Current KLA test: https://ieants.cc/kla/klatest/#/compare

[dariogoetz]

I'll try and see, if I can write some KLAnext-style metrics and generate a version that scores more closely to KLAnext.

Is KLAtest the same as KLAnext?

[iandoug]

Is KLAtest the same as KLAnext?

KLAnext is based on KLAtest but fixes some bugs and changes the scoring, finger weights, etc.

Let me see if I can send you the changed files. It might be difficult to integrate "simulated typing" as a metric, the logic likely somewhat different to the way other metrics work.

[dariogoetz]

That would be awesome.

My thinking was (having had a look at the kla code), that if the "simulated typing" only looks at the previous key without any relevant state otherwise, it should be sufficient to work with bigrams.

[iandoug]

I think I may need to take another look at my layout generation logic ... it does not appear to have generated any of the UCIEA variants (which I made by hand) and I think in theory it should have.

https://www.keyboard-design.com/letterlayout.html?layout=uciea-vanilla.en.ansi

Problem is that last time it ran for a month ... and can't do that now because we are having frequent load shedding/power cuts.

Let me stare at the code.

Busy running the 500 mill through your evaluator, done 179.8 million, found about 570 with scores below 253 which was my cut-off point for the 22 million ... and that found 980 in the 22 mill.

Maybe the second half will be better :-) Best so far on KLA is just a bit better than colibius-august.

Probably it will run faster if I skip the evaluation / scoring stuff and just generate, and then let your code do the evaluation. Will first see if there are any silly logic errors in the generator.

[dariogoetz]

I made some progress on the KLANext metrics: Have a look at the klanext branch. It contains the ported versions of the three metrics used on the klanext website (https://klanext.keyboard-design.com): same finger. same hand. and distance.

I used the browser debugger to try and make sense of the KLANext code to mimic it as best as possible. Since my analyser works a little differently, you need to add a leading space " " to a corpus file so that the first character is treated properly. Similarly, you should add a trailing space in the very end of the corpus (because KLANext brings the fingers back to the home row in the end); this however adds an extra cost of one keypress and -release on the thumbs (which can be neglected on sufficiently large corpora).

Finally, the resulting values are not perfectly in line with the KLANext analyzer. I am not perfectly sure what the reason is, but I believe it is the analysis.tmp.prevFinger (and variants for hands, key index, ...) in KLANext that can not keep up if there are multiple fingers involved simultaneously (e.g. shift following shift). Take for example the text "B)" on qwerty. It should involve a "same finger" usage on the right pinky (once for the shift for "B" and then for the "0" to generate the ")". This is not recognized by KLANext. But maybe, I misunderstand the KLANext code/metrics.

All in all, I would say that the ported KLANext metrics will fit well enough for your purposes. You should try and evaluate your layouts with that version once again.

[iandoug]

Hi

Finally, the resulting values are not perfectly in line with the KLANext analyzer. I am not perfectly sure what the reason is, but I believe it is the analysis.tmp.prevFinger (and variants for hands, key index, ...) in KLANext that can not keep up if there are multiple fingers involved simultaneously (e.g. shift following shift). Take for example the text "B)" on qwerty. It should involve a "same finger" usage on the right pinky (once for the shift for "B" and then for the "0" to generate the ")". This is not recognized by KLANext. But maybe, I misunderstand the KLANext code/metrics.

That's great, thanks... you make me look incompetent since I have not made any progress on my port. In the mean time I have just finished running the 500 million lines your your analyzer, and identified 2182 layouts that score below 253. Lowest was 242.57. Have been testing these on KLA, with RightThumb, need to do them all again with LeftThumb. Taken a week ... we still have black-outs which get in the way.

I discovered the problem with my generated layouts ... in an effort to reduce the search space, I created a "starter set" consisting of combinations of LeftIndex, RightnIndex, and RightPinky, and then found variations for the other fingers.

However, to keep the starter set small, I only used IndexFingers where the Clash Potential (basically a measure of SFB) was below 15, which turns out to be way too low, since it forces more effort onto the other fingers, including left pinky.

So I need to try again with different starting points, probably just the pinkies. Last time the generating step ran for a month (including messy evaluation steps)....

I will test your new version. My plan with my version was to start with small corpus file of 1k, then take the 50% of layouts that score best, do again with 2k file, and repeat process until only few hundred doing large corpus files. Layouts that do well with large files also do well with small files. I have already generated the various size corpus files, will add leading and tailing spaces.

Thanks for picking up a possible bug in KLA ... I did fix a few, but it is likely some remain ... will take a look at your example.

Cheers, Ian

[iandoug]

Take for example the text "B)" on qwerty. It should involve a "same finger" usage on the right pinky (once for the shift for "B" and then for the "0" to generate the ")".

At the moment I agree with you ... let me stare at the code. A bug here would also likely affect similar moves on AltGr.

Maybe the logic is like this... order of keypresses is:

RightShift B LeftShift 0 Thus strictly speaking 0 did not immediately follow RightShift. This is a subtle use case ... Does Rightshift return Home first, or go directly from Shift to 0 ....

However ... distance in cm for B) for right pinky is 18.93, while doing one letter at a time is 10.68+8.96 = 19.64 so it looks like pinky is going directly from shift to 0, thus should be added to SFB count....

Will stare at JS later ... this is going to shake up a lot of things :-) Possibly one of my fixes broke this ... but there were other issues re the SFBs etc as well.

Thanks, Ian

[dariogoetz]

Yes, it seems so. The code goes through the keys one after the other: prev "Shift" release -> "Shift" press -> prev "AltGr" release -> "AltGr" press -> prev "NumPad" release -> "NumPad" press -> "Actual Key". Every time, the analysis.tmp.prevFingerUsed (and analysis.tmp.prevHandUsed ...) is overwritten (both for "pressing" and "releasing") if applicable. That means that after a Shift key was involved, all keys involved for the previous character are "forgotten". In our example the history of analysis.tmp.prevFingerUsed is:

1. RightPinky (press shift)
2. LeftIndex (press b)
3. RightPinky (release shift, from prevShiftInfo)
4. LeftPinky (press shift)
5. RightPinky (press0``)

This somehow makes sense as the shifts usually are pressed before and released after the keys that shall be shifted. However, I would still consider those keys sufficiently close together (in time) to be considered a consecutive press.

Technically, for the consecutive finger presses, each one of them (in the enumeration above) is always compared to the next one (2. to 1., 3. to 2., 4. to 3., and 5. to 4.) and then overwritten. In particular the alleged consecutive finger 3. to 5. (or 1. to 5.) is not checked for (and cannot be checked as the value is always overwritten and the previous one lost).

This is also relevant for the consecutive hand metric.

[dariogoetz]

If you started already: I just pushed an almost 2x performance improvement (and a small fix) for the KLANex metrics to the klanext branch. You should use that if you evaluate the big set.

If I read the number correctly, it is now something like 1 (chained_english_0) to 1.5 (chained_english_7) milliseconds per evaluation.

Edit: Even less... it evaluates 100,000 layouts in about 11 seconds for chained_english_0 and in about 36 seconds for chained_english_7 - on one core.

[iandoug]

Thanks, finished earlier today. It ran at 100% CPU but the fans did not kick in. Did 100k with 1kB corpus in about 6 seconds. The 1MB corpus took about 45 seconds, versus about 1.5 minutes for your original non-KLA version. I have Ryzen 7 in this box.

The evaluations are more similar to KLA than other version, correlation not perfect though, but was able to reduce the 500 million down to 64 in about 25 steps.

Best per your eval is -,omjvphuxz=saetdfcrinb/.'kqwgly; with the [] swapped to the top row. Currently doing tests on KLA and will then run my other tests.

Your eval seems to use right thumb.

Layout above scores 4th on my default list on KLAnext. Which is probably the best possible, given that my index fangers were too constrained. So as a proof-of-concept for the method of getting a good layout programmatically, I consider it i success ... thanks for your help.

I will do it all again with different starting sets to find better layout.

Comparison of best per your eval, verses Poqtea, which scores higher on KLA. Poqtea is partial result at top.

I did modify your configs slightly. It seemed to make no difference if I used your finger penalties or mine though.

[iandoug]

FYI. The layout that came out on top was not one of those that scored best on your evaluation. The one in screenshot above actually did best on last few rounds while reducing the volumes ... and a close variant was also a contender.

Overall result is based on English, code, Proglish, plus other word-based metrics. These generated layouts took 20 of the top 30 positions in "Conventional + QWERTY shift pairs", and 18 of the top 30 in "Conventional", so rather impressive overall.

Let's see if Github handles large attachments...print to PDF not that great.

Keyboard-Design.com - Top 30 Best ANSI layouts (conventional + QWERTY shift pairs).pdf

Now I need to remove all except best one or two ..I actually like that homerow in image above since it is unique ... so will likely keep that one as well... or best one with that home row.

[iandoug]

BTW congrats on the speed-up. Amazing :-)

Will use it on the next batch.