Problems with existing font when using "Mathematical alphanumerical symbols"

[RolandSalz]

Unicode "Mathematical alphanumerical symbols" can be used within maxima symbols, which is a good thing and should definitely be kept. They contain Mathematical Bold, Fraktur Letters, Script Letters, etc. But they are very poorly represented (I speak for windows 7) in wxmaxima: way to small and shifted downward. Also, most of the Mathematical Symbols from the wxmaxima table are represented way to small, almost undistinguishable. These symbols could be valuable e.g. for use as user-definded infix operators. Is it possible to use another font for wxmaxima that does not have these problems?

[KubaO]

I could classify this as a feature request, since we'd be working around platform bugs (unless they are wxWidgets bugs)? In any case, this needs investigation before bug vs. feature request decision can be made. And we'd need Windows 7 VM. But maybe it reproduces on Windows 10, in which case it'd clearly be a bug.

@RolandSalz Do you still see this problem in the nightly, on whatever platform? Font handling has been significantly overhauled within wxMaxima, so this issue might have "fixed itself" in the meantime.

I'll close, but please reopen if you'll have more information about this behavior in the current nightly version.

[gunterkoenigsmann]

I hate font issues: In some countries in some windows versions the standard fonts came without ordinary, rounded parenthesis - and there are fonts that don't provide the standard ascii letters...

[KubaO]

I hate to spoil the party but the font files don’t change between Windows language versions as far as I know. The problems stem mostly from bugs in wxWindows I imagine, and in how wxMaxima uses it, and the fact that wxWindows couples the user too tightly to the quirks of the platform. It’s the sort of a problem that affects mostly wxWindows, since it doesn’t offer a single, tested, canonical way of representing strings. So it does various things based on how it’s configured for the build and absurd bugs result from that.

One thing Qt got right in IIRC Qt 2 was to settle on UTF-16 and make that the only way of dealing with strings. And no “ANSI” builds for Windows - that was totally unnecessary since times of Windows NT 4, for crying out loud. Once you drop the idiocy of “ANSI” (and worse - multibyte character sets), it’s rather easier on Windows. But wxWidgets seems stuck with C++98 and some very legacy applications they have to support. Yuck.

The only 8-byte “exception” is the need for transparent filename representation on Unix. Because on Unix filenames are mostly zero-terminated byte arrays where the only special character is the slash and the periods as long as they are between slashes. So any Unix system will happily handle a filename or path that is absolutely not valid UTF-8 and not really valid in any encoding. Qt tries to translate it to canonical UTF-16 and can fail, unable to deal with some badly mangled filenames that have junk in them, whereas the syscalls and glibc are OK with it. And conversely on Windows you can have similar junk in the 16-bit filenames - it’s not validated at all to be valid UTF-16, just as it isn’t checked to be UTF-8 on Unix (that probably would slow down file access too much, maybe). And then some network file systems (NFS and others) pass the names without constraints, some others will validate to UTF-8 or 16 and fail otherwise etc. It’s a mess, but thankfully fonts don’t succumb to it.

10 juli 2020 kl. 9:42 em skrev Gunter Königsman notifications@github.com:

 I hate font issues: In some countries in some windows versions the standard fonts came without ordinary, rounded parenthesis - and there are fonts that don't provide the standard ascii letters...

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

[gunterkoenigsmann]

I don't know if the problem arises in official versions of windows and if these users found a way to change the default don't manually. But there are many Asian computers on which the letters a-z and the opening and closing parenthesis are missing in the default font and the NTFS on the windows computer I sometimes manage to borrow encodes German umlauts with Codepage 850 while it encodes Cyrillic letters as either utf16 or Unicode: at least as something SBCL knows how to deal with. I once read that NTFS can be instructed to save a few bytes by using codepage850 in filenames at creation time.

wXMaxima is important enough that we end up with reports from many users with real exotic system setups...

[KubaO]

I can assure you that these glyphs exist there as well if they exist on non-Asian localizations. It’s possible that wxWidgets font handling and/or encoding that fails. Code pages have no meaning for UTF-16 applications. In Windows, the code page affects the 8-bit or multibyte encoding. Once you use the “Unicode” (...W) WINAPI, there’s only one encoding and code pages and such nonsense is out of the picture. wxWidgets also has this idiotic notion that a font has “encoding”, whereas it’s up to the underlying platform to deal with that.

What most likely happens in wxMaxima is the inconsistent font defaulting system - every place a font is used deals with it differently. This got improved a bit already, but I have a branch where this is further fixed once and for all - where the users of styles modify nothing besides additional scaling in sub/superscripts/indices, and where font fallbacks are evaluated with full feedback to the user, ie when you open the Style pane, the font indicated is the font used, not a “wish” for a font. I know that it was a significant problem, and it explains a lot of font problems that wxMaxima had. The font API in wxWidgets doesn’t help here either.

Basically, at the moment the font selections in wxMaxima are a bit of a lie, since the code can change fonts at the point of use via defaulting, whereas this should never be happening. The defaulting all over the place is the next thing to go for the 20.08 release.

11 juli 2020 kl. 5:08 fm skrev Gunter Königsman notifications@github.com:

 I don't know if the problem arises in official versions of windows and if these users found a way to change the default don't manually. But there are many Asian computers on which the letters a-z and the opening and closing parenthesis are missing in the default font and the NTFS on the windows computer I sometimes manage to borrow encodes German umlauts with Codepage 850 while it encodes Cyrillic letters as either utf16 or Unicode: at least as something SBCL knows how to deal with. I once read that NTFS can be instructed to save a few bytes by using codepage850 in filenames at creation time.

wXMaxima is important enough that we end up with reports from many users with real exotic system setups...

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

[gunterkoenigsmann]

Defaulting was introduced because there were many big tickets caused by users first selecting a font in wxMaxima and then uninstalling it, for example by upgrading MacOs. The result often was a crash as working with invalid fonts wasn't really tested, and often this happened before the user could select a new font. A fallback to a default don't helped in these cases.

[KubaO]

That’s of course understandable - I know that such things happen because there’s a need for them. It’ll just be that the defaulting is done visibly and once: when you open Config->Style, you’ll see exactly the font currently in use as default and math, ie the actual name substituted due to defaulting. And we’ll also have a central place where we can test fonts for functionality, ie. check if they have given characters etc.

The font cache now caches given font on both ends of the selection process: first it gets the Style key for the font request, then if the request fails it caches the negative result. If it succeeds, it stores the font under that key, then gets actual properties of the font into another Style, and if it differs then it caches the same font under that other key. That’s what you see in debug pane: cache font misses may have two lines - one as a request, one as what it resolves to.

On Windows, default font requests have different encoding than the actual selected font, so they get cached under both keys.

11 juli 2020 kl. 4:37 em skrev Gunter Königsman notifications@github.com:

 Defaulting was introduced because there were many big tickets caused by users first selecting a font in wxMaxima and then uninstalling it, for example by upgrading MacOs. The result often was a crash as working with invalid fonts wasn't really tested, and often this happened before the user could select a new font. A fallback to a default don't helped in these cases.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.