search

amwx / FluentAvalonia

Control library focused on fluent design and bringing more WinUI controls into Avalonia
MIT License
904 stars 86 forks source link

Planned changes to Symbol enum & font #321

Closed amwx closed 1 year ago

amwx commented 1 year ago

Some updates/fixes coming to the Symbol enum and font. Primary goal here is to cut down on repetitive icons and icons without glyphs.

image

Symbols in green will be added to the font & have Obsolete removed. Symbols in yellow and grey still don't exist, but could probably be made manually in inkscape (or any vector editing app, mostly reusing the existing icons), need to figure out if the system that's used to build the font would support this. Note that this is only an option at this point. Placeholder will be removed

image

I believe most of these are placed in a special area in the font specifically for FluentAvalonia. If so, they will likely be removed from the font too.

Within the actual font, various icons that didn't previously have glyphs, now do and will be added to the font image

robloo commented 1 year ago

@amwx Do you have your own tooling for this now or do you plan to modify the font file directly? Otherwise, I can get the build to work again here:

There were stability issues with font source file name/path breaking changes upstream (and worse, files just being deleted) so I pull in most sources files now and there is a helper script for it. This was mostly done but I still had a few fixes I think to source glyph locations in some of the mappings files, not sure if FluentAvalonia.json will build as-is right now.

Just let me know if you want help with this or for me to make the updates.

Also keep in mind it was nice for the symbols enum to be built automatically from the font/mappings file itself. There is a small method to auto-generate that.

Edits for specific points above

Several icons have incorrect glyphs, will be corrected

I'm curious which ones you see problems with. Then I would read the mapping notes. Several were "as close as possible" but are clearly marked as a placeholder because no Fluent UI System icon existed at the time. Those that have upstream icons now could be fixed.

need to figure out if the system that's used to build the font would support this.

Concerning custom icon sources (in .svg files), that was always planned but was never needed. The Uno Platform ended up just manually building their fonts and not releasing their custom SVG files. So it is "half implemented" though and could be completed.

I believe most of these are placed in a special area in the font specifically for FluentAvalonia. If so, they will likely be removed from the font too.

Yes, FA-specific icons are still in the private use area starting at 0xF8000 (pushed high enough to avoid any future collisions with Fluent UI System [hopefully]). See below for all notes on that.

           // Initialize the starting value of all new symbols Unicode points here.
            // New symbols are those that do not exist by name in either SegoeFluent or the
            // WinUI Symbol enum.
            // The starting Unicode point here must be within range:
            //   U+F0000..U+FFFFD (Supplementary Private Use Area-A)
            // This ensures it is outside the range of values used by SegoeFluent/SegoeMDL2Assets.
            // Additionally, it must be high enough to almost certainly never overlay with
            // the FluentUISystem (currently ending at 0x10--- within ).
            uint newUnicodePointStart = 0xF8000;
amwx commented 1 year ago

@amwx Do you have your own tooling for this now or do you plan to modify the font file directly? Otherwise, I can get the build to work again here:

* https://github.com/robloo/SymbolIconManager/blob/main/Source/Models/Specialized/FluentAvalonia.cs

* https://github.com/robloo/SymbolIconManager/blob/main/Source/Data/Mappings/FluentAvalonia.json (updating the mappings file is most important)

There were stability issues with font source file name/path breaking changes upstream (and worse, files just being deleted) so I pull in most sources files now and there is a helper script for it. This was mostly done but I still had a few fixes I think to source glyph locations in some of the mappings files, not sure if FluentAvalonia.json will build as-is right now.

Just let me know if you want help with this or for me to make the updates.

There are several glyphs which have been moved/renamed/codepoint changed in Fluent UI, as they're showing up blank when looking at my mapping file so you're probably right this wouldn't build right now. An example is the Audio icon, currently mapped to FE2A, where the icon now lives at E854

Right now I'm working my way through the mapping file making the corrections. Once I get that done, is it just as simple as popping over to the 'Fonts' tab and hitting 'Build FluentAvalonia font'?

I also remember last year we had to make some changes to the font so it would render correctly in Avalonia. Did you manage to get them into the build process or was there some manual work left over? I can't remember.

Also keep in mind it was nice for the symbols enum to be built automatically from the font/mappings file itself. There is a small method to auto-generate that.

I'm still looking over the source to see how it all works, but is there a flag or something that controls whether the icon ends up in the Symbol enum?

Edits for specific points above

Several icons have incorrect glyphs, will be corrected

I'm curious which ones you see problems with. Then I would read the mapping notes. Several were "as close as possible" but are clearly marked as a placeholder because no Fluent UI System icon existed at the time. Those that have upstream icons now could be fixed.

These are the icons I've identified and the reason. It's only a handful and most of them are just minor things image

need to figure out if the system that's used to build the font would support this.

Concerning custom icon sources (in .svg files), that was always planned but was never needed. The Uno Platform ended up just manually building their fonts and not releasing their custom SVG files. So it is "half implemented" though and could be completed.

Not a big deal right now, mostly just a thought. Its not that many icons to make it worthwhile right now.

robloo commented 1 year ago

There are several glyphs which have been moved/renamed/codepoint changed in Fluent UI, as they're showing up blank when looking at my mapping file so you're probably right this wouldn't build right now. An example is the Audio icon, currently mapped to FE2A, where the icon now lives at E854

Let me go through this weekend and get the build to use local glyph icon sources. Then I can fix or remove the icons that are outdated in the core Microsoft font mapping files. (These are also used as the base for FluentAvalonia's font). Everything is being pulled into the repo now (just needs to be properly used in code) so I don't need to worry about upstream breaks in the future the whole thing it too brittle as it is now.

It might be best to wait until I can get that all fixed up before you change too much in the FluentAvalonia.json mapping file. Some icon mappings might not be bad -- just the glyph is pointing to an old file no longer available online. I should be able to get it to build in that situation though.

Once I get that done, is it just as simple as popping over to the 'Fonts' tab and hitting 'Build FluentAvalonia font'?

In theory, yes* (see below), but I need to update the base Microsoft symbol font mappings and fix those issues. FluentAvalonia.json should then be rebuilt and I can do that this weekend too. The rebuilt file can then be used for your updates.

I also remember last year we had to make some changes to the font so it would render correctly in Avalonia. Did you manage to get them into the build process or was there some manual work left over? I can't remember.

Unfortunately, I don't think the python scripting issue was ever fixed in FontForge so the manual step is still required: https://github.com/robloo/SymbolIconManager/issues/1#issuecomment-1060234058. It should be as simple as matching the first screenshot. The font builder python script itself has all of the notes and details for what it is needed. This is always a good reference to figure out what needs to be done. I tried to document it for myself in the years to come as well in case something needs to be adjusted.

https://github.com/robloo/SymbolIconManager/blob/e3a8a987eb19f4151edd2a2eaf783c6b9ddec628/Source/Models/FontBuilder.cs#L344-L369

I'm still looking over the source to see how it all works, but is there a flag or something that controls whether the icon ends up in the Symbol enum?

There is a CheckBox in the UI that controls this. It will just generate an enum value for every mapping in the FluentAvalonia.json file.

amwx commented 1 year ago

Let me go through this weekend and get the build to use local glyph icon sources. Then I can fix or remove the icons that are outdated in the core Microsoft font mapping files. (These are also used as the base for FluentAvalonia's font). Everything is being pulled into the repo now (just needs to be properly used in code) so I don't need to worry about upstream breaks in the future the whole thing it too brittle as it is now.

It might be best to wait until I can get that all fixed up before you change too much in the FluentAvalonia.json mapping file. Some icon mappings might not be bad -- just the glyph is pointing to an old file no longer available online. I should be able to get it to build in that situation though.

I started going through and updating just the icons that are now missing (just updating to the new Fluent UI code point) and got about 3/4 of the way through before the app crashed when I dragged the window (nothing specific to your app, some weird null ref somewhere in Avalonia that of course isn't repeatable). So that's fine, I'll wait for your changes before going after this.

robloo commented 1 year ago

I'm really not sure what they did in the Fluent UI System repo but it appears they made several breaking changes. Deleting glyphs and unicode point that previously existed -- perhaps even changing a few.

It is taking a bit to go through and rebuild everything... Still hoping once its done and font generation is done using local glyph sources it won't be this brittle for upstream changes again. Then perhaps I'll sync only once every few years.

robloo commented 1 year ago

@amwx FluentAvalonia.json is ready for your changes now. Everything also should be able to build offline using local SVG files in the repo itself. This hopefully means everything is stable going forward and it will always build as it does now.

All icon mappings should be fixed (the 3 base windows symbol font mappings) then I copied those fixes into FluentAvalonia.json. So the "icons needing update" should be done already. The only thing you have to do is delete the FA-specific icons you no longer want in the font. ONLY delete the ones in the FA-specific Unicode range above 0xF8000 though. Any changes you make elsewhere will be overwritten the next time the mapping is rebuilt from the base SegoeFluent information.

Avalonia itself has issues so the mappings editor doesn't always display correctly. There are several ListBox issues I noticed and also an issue with container recycling or images causing the display of incorrect glyphs even though the view model and mappings are correct. So be careful there -- it might be the data is correct just Avalonia is messing it up in the UI.

amwx commented 1 year ago

@robloo Thanks again for your work on this.

I've also updated these icons in the SegoeFluent.json mapping file, *most* are icons didn't exist previously that now exist in FluentUI. Some, I've added because something exists even if it isn't perfect. Sharing if you want to make changes too. The ones in purple were originally in my mapping file, but I updated them here too. There's also "Accounts" in the SegoeFluent.json file, which was using the 'at' symbol from LineAwesome, that is now in FluentUI at F53C (ic_fluent_mention_20_regular).

image

robloo commented 1 year ago

@amwx

I've also updated these icons in the SegoeFluent.json mapping file

Ok, thanks for finding these updates! It's also great that upstream has added more icons we can use and I can pull these in no-problem. Did you already update the JSON file(s) in a branch? I can accept a PR to change. Otherwise, I'll update the file myself and set glyph/metaphor match quality: https://github.com/robloo/SymbolIconManager/blob/main/Docs/IconMapping.md#match-quality. That's really useful to see what needs to be fixed or with heuristics when building fonts from multiple sources (choosing only the best match).

I don't know if you plan on keeping the FluentAvalonia.json file separate but I can also accept any changes you make to it upstream. It's better IMO to keep it all in one place but you are free of course to keep your own font-specific fork.

amwx commented 1 year ago

@robloo I'll let you handle SegoeFluent.json. I didn't pay much attention to comments or match qualities for my local copy, and that way you can decide if its a good choice for the general purpose rather than just my font file.

Question- How do you add an icon to the mapping file? I didn't see anything in the UI, so I manually added an entry in the json file with the destination unicode point set to 0000 and it crashes on line 174 in FluentAvalonia.RebuildMappings ("invalid symbol enum icon detected"). This is in the foreach for WinUISymbols.Symbol, weirdly its happening on the very first symbol Previous so I'm clearly missing something. Based on my understanding of that method, setting the destination unicode point to 0 should cause it to be rewritten with the next available FA unicode point.

this is what I added in FluentAvalonia.json:

{
    "Destination": {
      "IconSet": "Undefined",
      "UnicodePoint": "0000",
      "Name": "CodeHTML"
    },
    "Source": {
      "IconSet": "FluentUISystemRegular",
      "UnicodePoint": "F2EF",
      "Name": "ic_fluent_code_20_regular"
    },
    "GlyphMatchQuality": "Exact",
    "MetaphorMatchQuality": "Exact",
    "IsPlaceholder": false,
    "Comments": ""
  }
robloo commented 1 year ago

I'll let you handle SegoeFluent.json

Ok, I'll update that soon. It will be usable when building the FluentAvalonia font.

How do you add an icon to the mapping file? I didn't see anything in the UI

Yea, nothing was added for that. The reason is, before FluentAvalonia font, all fonts have a pre-defined list of icons. So all mappings were initialized using, for example, the SegoeFluent destination icon information that Microsoft decides.

For FluentAvalonia this was also originally generated using an algorithm. So bottom line, until now, it just hasn't been needed for anything other than to edit existing mappings.

That means its correct to just modify the mapping file by hand to add/remove specific icon mappings. Sorry the tool is a bit rough here and is only used in a few cases so havnt spent a lot of time making it fully general purpose. Actually, all the UI code is ugly and probably should be rewritten at some point (that wasn't the focus for a tool like this).

I will definitely fix whatever is going on with the crashes. It should work to add a new blank mapping and then rebuild to assign a UnicodePoint. Will take a look later today on that.

robloo commented 1 year ago

@amwx

I manually added an entry in the json file with the destination unicode point set to 0000 and it crashes on line 174 in FluentAvalonia.RebuildMappings ("invalid symbol enum icon detected").

I fixed this issue. Basically, this is the first time the FluentAvalonia mappings have been re-built with existing, custom UnicodePoints already assigned. So it was missing the logic needed to detect the max UnicodePoint already used and then start from there (not start from 0xF8000).

Also keep in mind the rebuild process will re-sort the mappings by destination icon name. This is to keep things stable but in practice it means any mappings you add at the end will be moved once a name is assigned.

Finally, I updated the project to .NET 7.0 so you might need to install the new SDK.

robloo commented 1 year ago

@amwx I've committed upstream all of the new icons (https://github.com/amwx/FluentAvalonia/issues/321#issuecomment-1483990359) you outlined above. Thanks so much for sharing these!

You should be able to rebuild the FluentAvalonia.json mappings in the future without loosing any information. I kept the icons as you wrote only clarifying the metadata a bit.

robloo commented 1 year ago

@amwx Finally, I updated the user guide with more instructions if you build this again in the future. Apologies that the special metrics setup step was not clear before. I guess you figured it out but it was a bit more complicated than I remember.

https://github.com/robloo/SymbolIconManager/blob/main/Docs/UserGuide.md