win32: UTF8_OUTPUT: speedup for big outputs

[avih]

With the native Windows console, writeCon_utf8 which converts a stream of UTF8 into console output is about 1.4x slower for big unicode writes than the native fwrite (e.g. when the console codepage is UTF8), which is not too bad.

However, newer versions of conhost are quicker, e.g. OpenConsole.exe (which is conhost) which ships with the Windows terminal is about 4x faster than the native conhost in processing (unicode?) input.

And when conhost can process inputs much quicker, it turned out that fwrite throughput was nearly 3x better than writeCon_utf8.

Luckily, this turned out to be mainly due to the internal 256 wide chars buffer which writeCon_utf8 uses, and that with 4096 buffer it becomes only ~ 10% slower than fwrite, which is much better.

However, making the console window very small such that it needs to spend very little time on rendering, makes it apparent that there's still a difference - writeCon_utf8 is about 30% slower than fwrite, but that's still not bad, and that's also an uncommon use case.

So this commit increases the buffer, and also allocates it dynamically (once) to avoid abusing the stck with additional 8K in one call.

[rmyorston]

It's in the latest prerelease. Thanks!

[avih]

Thanks.

I see you merged it as is rather than rebasing it on top of master - creating a non-linear history.

Any reason for that?

EDIT: I see it's the same with my other earlier PRs. FWIW, unless there's a good reason to not rebase first, I think a linear history is preferable.

The github "rebase and merge" button does create a linear history, and doing that locally instead of via the online github interface should work too.

I think the only place where non-linear history makes sense is when merging upstream busybox.