ctrlaltf24 commented 1 month ago

Reproduction steps:

Launch Logos 10 (v36.1.0.6) with wine-devel 9.12 (or 9.16)
Open a biblical word study for "worship" (some words cause this crash, others don't).
Expand the Hebrew words section
Application crashes with the error below

2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Application: Logos.exe\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.724.31311\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L".NET Version: 8.0.7\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Description: The application requested process termination through System.Environment.FailFast.\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Message: Unhandled exception in async work\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.CultureInfoUtility.CreateSpecificCulture(LanguageName language)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Globalization.Culture.CreateSpecificCulture()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Globalization.Culture.get_FormatCultureInfo()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Globalization.Culture.CreateStringComparer(CaseSensitivity caseSensitivity)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Globalization.Culture.get_StringComparerIgnoreCase()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.StopListUtility.IsStopLemma(LemmaReference l)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`2 predicate)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.EnumerableUtility.TrimEndWhere[T](IEnumerable`1 seq, Func`2 fnPredicate)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.EnumerableUtility.ToReadOnlyCollection[T](IEnumerable`1 seq)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.StopListUtility.TrimLemmas(IEnumerable`1 lemmas)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionUtility.GetJoinedLemmasForTranslation(ReverseInterlinearResource resource, String strWord, ReferenceRange referenceRange, IWorkState state)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionUtility.GenerateDataPointsAndSparklineForTranslation(String word, IEnumerable`1 reverseInterlinears, ResourceLists resourceLists, ReferenceRange referenceRange, IWorkState state, ReadOnlyCollection`1& dataPoints, SparklineDat"...
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.Guides.Sections.WordLemmaTranslationSectionSource.GenerateData(WordGuideKey keyWord, LibraryCatalog libraryCatalog, ResourceLists resourceLists, ReferenceRange referenceRange, IWorkState state)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.Guides.Sections.WordLemmaTranslationSectionSource.GenerateData(WordGuideKey keyWord, String strResourceOverride, String strResourceAbbreviatedTitle, ResourceManager resourceManager, VersifiedResourceManager versifiedResourceManager, ReverseInterlinearManager "...
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionBase.DoWork(TranslationSectionBase section)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.Threading.AsyncWorkerTask`1.EnumMoveNext()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Stack:\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Environment.FailFast(System.String, System.Exception)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.ApplicationUtility.FailFast(System.String, System.Exception)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.Threading.AsyncWorkerTask`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].EnumMoveNext()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.Threading.AsyncWorkerTask`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ContinueExecution(System.Object)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at Libronix.Utility.Threading.GroupedThreadPool.ExecuteNextCallback(System.Object)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Threading.QueueUserWorkItemCallback.Execute()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Threading.ThreadPoolWorkQueue.Dispatch()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()\n"

thw26 commented 1 month ago

See #156, which has some reports of this same issue. Those posting had a few issues going on, so it would be good to keep this open separately.

ptrotter1 commented 1 month ago

Another way to crash with the same error is to open the "Reverse Interlinear Pane" in a Bible.

thw26 commented 1 month ago

Thanks, that should make it very easy to test.

NichCrusie commented 1 month ago

Commenting for visibility. I also experienced this yesterday evening, but for various other words.

ctrlaltf24 commented 1 month ago

Looking into this in more depth...

Found:

This appears to be the failing GetCultureInfo call https://github.com/dotnet/runtime/blob/v8.0.7/src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.Icu.cs#L475
Which is populated from https://github.com/dotnet/runtime/blob/v8.0.7/src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.Icu.cs#L453
(probably) https://github.com/dotnet/runtime/blob/2aade6beb02ea367fd97c4070a4198802fe61c03/src/native/libs/System.Globalization.Native/pal_locale.c#L156
uloc_countAvailable defined in icu repo

Open questions:

(unlikely) Is it possible this is a problem with the .NET runtime under wine? Similar to: https://github.com/dotnet/runtime/blob/6d23f64952d0f242c4dcc7cbb230a619be930309/src/native/libs/System.Globalization.Native/pal_locale.m#L720 . There are several issues listed there for en-us-posix https://github.com/dotnet/runtime/issues?q=%22en-us-posix%22%20
(general) What's a good way to debug Logos w/out violating the terms (https://www.logos.com/terms 4b "Reverse engineer, disassemble, decompile, make any attempt to discover or use the source code of any Services, or make any attempt to “unlock” or circumvent the digital copyright protection of any Services;")

Failed attempts at producing a different result:

changing language in Logos to Spanish, same backtrace, (en-us-posix)
set LC_ALL="ja_JP" and LANG="ja_JP.UTF8", same backtrace
enabled WINEDBG=+keyboard, same result, no relevant logs
installing locales-all package, same result
removing generated C/POSIX locale files (DO NOT DO THIS ON YOUR SYSTEM) - still was detected as supported
tried inspecting running envs for all processes with "Logos" in the name, all include the LANG env var. Checking to make sure it isn't falling back to posix via https://www.gnu.org/software/libc/manual/html_node/Standard-Locales.html
removing the en_US_POSIX locale from icudtl.dll via icupkg tool
(long shot, didn't expect to work) copying the en_US_POSIX locale to en-us-posix via icupkg tool and mv

Observations:

en-us-posix seems to be the source of a number of bugs in icu, however none of these issues seem to relate to our issue at this time https://unicode-org.atlassian.net/issues/?jql=text%20~%20%22en-us-posix%22

ctrlaltf24 commented 1 month ago

@thw26 would it be possible to document how the icu release was created (or update script)?

I'm stuck regarding looking into this issue due to my inability to get an icu I compile to be loaded by logos. I don't understand how icu shared library is getting generated. I'm not that familiar with windows build systems, but did figure some of it out. New to building on windows but have done a fair bit of development on linux systems. It should be noted I'm using MSVC build tools v2019 and windows 11 to compile. Trying again with linux and gcc..

unknown source

C:/windows/system32/icu.dll (there is no .dll with roughly that name, nor roughly that size I could find doing a clean build of icu, what is this file?) Looking at icuuc.dll, it looks like icu.dll contains the common icu library (icuuc just forwards to it).
found correlation between:
icu/icu4c/sources/lib/icuin76.dll and C:/windows/system32/icuin.dll
icu/icu4c/sources/lib/icuout76.dll and C:/windows/system32/icuout.dll
icu/icu4c/sources/data.out/build/icudt76l/ and C:/windows/globalization/ICU/*.res
C:/winows/globalization/ICU/icudtl.dat and icu/icu4c/sources/data/tmp/icudt761.dat

n8marti commented 1 month ago

We copied the v75.1 release from https://github.com/unicode-org/icu/releases into https://github.com/FaithLife-Community/icu/releases, but I see that we have icu-win.tar.gz, while I don't see that exact file in the unicode-org repo's release. @thw26, can you add any details to where our file comes from vs what's available in the original repo? It seems like it would be good to rule out any issues that could have gotten introduced by our release.

Once the file is downloaded, we just uncompress it and drop its contents straight into drive_c:

def install_icu_data_files(app=None):
    releases_url = "https://api.github.com/repos/FaithLife-Community/icu/releases"  # noqa: E501
    json_data = network.get_latest_release_data(releases_url)
    icu_url = network.get_latest_release_url(json_data)
    # icu_tag_name = utils.get_latest_release_version_tag_name(json_data)
    if icu_url is None:
        logging.critical("Unable to set LogosLinuxInstaller release without URL.")  # noqa: E501
        return
    icu_filename = os.path.basename(icu_url)
    network.logos_reuse_download(
        icu_url,
        icu_filename,
        config.MYDOWNLOADS,
        app=app
    )
    drive_c = f"{config.INSTALLDIR}/data/wine64_bottle/drive_c"
    utils.untar_file(f"{config.MYDOWNLOADS}/icu-win.tar.gz", drive_c)

The installer checks to see if the ICU license file exists or not before installing the ICU files:

    icu_license_path = f"{config.WINEPREFIX}/drive_c/windows/globalization/ICU/LICENSE-ICU.txt"  # noqa: E501
    if not utils.file_exists(icu_license_path):
        wine.install_icu_data_files(app=app)

so you could theoretically short-circuit that process by touching that file, then going back and dropping in any other ICU files after the installation completes to test how they work. Or just simply replace the ones we install with other versions.

ctrlaltf24 commented 1 month ago

Looked into it a little more

Two zip differences (between upstream icu and ours):

ICU packages 32bit/64 bit separately, we publish both (one in system32, the other in syswow64), good thinking!
ICU packages with renaming on (all functions smurfed to end in the version number), we don't (corresponding to the --disable-renaming flag on ./configure)
(trivial) they publish as a .dll, we publish as a .dat (both are supported by icu)
different directory structure (as noted on the FaithLife-Community/icu release)
metaZones.rs,timezoneTypes.res,windowsZones.res and zoneinfo64.res copied from build process (not present in upstream package). Guessing there was a need for these, so they were added
new dll called icu.dll (appears to be a combined implementation, unsure how to build this)

icuuc.dll doesn't contain implementations, just pointing to icu.dll

$ dumpbin /exports icu-win\windows\syswow64\icuuc.dll
...
    534  215          utf8_nextCharSafeBody (forwarded to icu.utf8_nextCharSafeBody)
    535  216          utf8_prevCharSafeBody (forwarded to icu.utf8_prevCharSafeBody)
    536  217          utrace_format (forwarded to icu.utrace_format)
    537  218          utrace_functionName (forwarded to icu.utrace_functionName)
    538  219          utrace_getFunctions (forwarded to icu.utrace_getFunctions)
    539  21A          utrace_getLevel (forwarded to icu.utrace_getLevel)
    540  21B          utrace_setFunctions (forwarded to icu.utrace_setFunctions)
    541  21C          utrace_setLevel (forwarded to icu.utrace_setLevel)
    542  21D          utrace_vformat (forwarded to icu.utrace_vformat)

icuin.dll doesn't contain implementations either, just points to icu.dll

$ dumpbin /exports icu-win\windows\syswow64\icuin.dll
...
    408  197          utrans_openU (forwarded to icu.utrans_openU)
    409  198          utrans_register (forwarded to icu.utrans_register)
    410  199          utrans_setFilter (forwarded to icu.utrans_setFilter)
    411  19A          utrans_toRules (forwarded to icu.utrans_toRules)
    412  19B          utrans_trans (forwarded to icu.utrans_trans)
    413  19C          utrans_transIncremental (forwarded to icu.utrans_transIncremental)
    414  19D          utrans_transIncrementalUChars (forwarded to icu.utrans_transIncrementalUChars)
    415  19E          utrans_transUChars (forwarded to icu.utrans_transUChars)
    416  19F          utrans_unregisterID (forwarded to icu.utrans_unregisterID)

I'm not familiar enough with cpp or windows build systems to figure out how to build the combined library

thw26 commented 1 month ago

We store it in .tar.gz since we have an untar command already made.

    utils.untar_file(f"{config.MYDOWNLOADS}/icu-win.tar.gz", drive_c)

ctrlaltf24 commented 1 month ago

Sure I realize that, was more wondering how icu.dll/icuuc.dll and icuin.dll were created. Were they copied from a windows install of Logos?

Checked against my windows logos install and confirmed icu.dll/icuin.dll/icuuc.dll all matched the format we ship. (as documented here https://learn.microsoft.com/en-us/windows/win32/intl/international-components-for-unicode--icu-#version-1903-may-2019-update)

ctrlaltf24 commented 1 month ago

I attempted to open the icudtl.dat that Logos for windows ships with:

increased the STRING_STORE_SZIE in tools/toolsutil.h by 100x (can't open the file without it)

building the icupkg tool on linux

$ bin/icupkg ~/QubesIncoming/Win11Logos/icudtl.dat 
icupkg: input .dat item name "icudt72l/aa.res" does not start with "icudtl/"

I would if they were building with ICU version 72 (rather than the 76 we ship with). Unsure if it matters

Tried to run Logos on linux with the dlls from the windows install, failed during logos startup (same thing happens if I use the .dlls from nuget)

2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"Stack:\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CalendarData+<>c.<GetCalendarInfo>b__32_0(System.Span`1<Char>, System.String, System.Globalization.CalendarId, System.Globalization.CalendarDataType)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at Interop.CallStringMethod[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Globalization.CalendarId, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Globalization.Cal"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CalendarData.GetCalendarInfo(System.String, System.Globalization.CalendarId, System.Globalization.CalendarDataType, System.String ByRef)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CalendarData.IcuLoadCalendarDataFromSystem(System.String, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CalendarData.LoadCalendarDataFromSystemCore(System.String, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CalendarData..ctor(System.String, System.Globalization.CalendarId, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CultureData.GetCalendar(System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CultureData.DateSeparator(System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.DateTimeFormatInfo.InitializeOverridableProperties(System.Globalization.CultureData, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Globalization.CultureInfo.get_DateTimeFormat()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Controls.DateTimeHelper.GetDateFormat(System.Globalization.CultureInfo)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Controls.DateTimeHelper.GetCurrentDateFormat()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Controls.Calendar..cctor()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Runtime.CompilerServices.RuntimeHelpers.RunClassConstructor(System.RuntimeTypeHandle)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.WindowsBase.SecurityHelper.RunClassConstructor(System.Type)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.DependencyProperty.FromName(System.String, System.Type)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.Baml2006Reader.WriteTypeConvertedInstance(Int16, Int32)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.Baml2006Reader.Process_PropertyCustom()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.Baml2006Reader.Process_OneBamlRecord()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.Baml2006Reader.Process_BamlRecords()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.Baml2006Reader.Read()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.TransformNodes(System.Xaml.XamlReader, System.Xaml.XamlObjectWriter, Boolean, Boolean, Boolean, System.Xaml.IXamlLineInfo, System.Xaml.IXamlLineInfoConsumer, MS.Internal.Xaml.Context.XamlContextStack`1<System.Windows.Markup.WpfXamlFrame>, System.W"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.Load(System.Xaml.XamlReader, System.Xaml.IXamlObjectWriterFactory, Boolean, System.Object, System.Xaml.XamlObjectWriterSettings, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.LoadBaml(System.Xaml.XamlReader, Boolean, System.Object, System.Xaml.Permissions.XamlAccessLevel, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.XamlReader.LoadBaml(System.IO.Stream, System.Windows.Markup.ParserContext, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Application.LoadBamlStreamWithSyncInfo(System.IO.Stream, System.Windows.Markup.ParserContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.AppModel.AppModelKnownContentFactory.BamlConverterCore(System.IO.Stream, System.Uri, Boolean, Boolean, Boolean, Boolean, System.Windows.Markup.XamlReader ByRef, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.AppModel.MimeObjectFactory.GetObjectAndCloseStreamCore(System.IO.Stream, MS.Internal.ContentType, System.Uri, Boolean, Boolean, Boolean, Boolean, System.Windows.Markup.XamlReader ByRef, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.ResourceDictionary.set_Source(System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.WpfSharedBamlSchemaContext+<>c.<Create_BamlProperty_ResourceDictionary_Source>b__344_0(System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Baml2006.WpfKnownMemberInvoker.SetValue(System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.Xaml.Runtime.ClrObjectRuntime.SetValue(System.Xaml.XamlMember, System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.Xaml.Runtime.ClrObjectRuntime.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at MS.Internal.Xaml.Runtime.PartialTrustTolerantRuntime.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlObjectWriter.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlObjectWriter.Logic_ApplyPropertyValue(MS.Internal.Xaml.Context.ObjectWriterContext, System.Xaml.XamlMember, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlObjectWriter.Logic_DoAssignmentToParentProperty(MS.Internal.Xaml.Context.ObjectWriterContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlObjectWriter.Logic_AssignProvidedValue(MS.Internal.Xaml.Context.ObjectWriterContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlObjectWriter.WriteEndMember()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Xaml.XamlWriter.WriteNode(System.Xaml.XamlReader)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.TransformNodes(System.Xaml.XamlReader, System.Xaml.XamlObjectWriter, Boolean, Boolean, Boolean, System.Xaml.IXamlLineInfo, System.Xaml.IXamlLineInfoConsumer, MS.Internal.Xaml.Context.XamlContextStack`1<System.Windows.Markup.WpfXamlFrame>, System.W"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.Load(System.Xaml.XamlReader, System.Xaml.IXamlObjectWriterFactory, Boolean, System.Object, System.Xaml.XamlObjectWriterSettings, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.WpfXamlLoader.LoadBaml(System.Xaml.XamlReader, Boolean, System.Object, System.Xaml.Permissions.XamlAccessLevel, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Markup.XamlReader.LoadBaml(System.IO.Stream, System.Windows.Markup.ParserContext, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at System.Windows.Application.LoadComponent(System.Object, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at LDLS4.OurApp.InitializeComponent()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at LDLS4.OurApp.InitializeAndRun(System.Collections.ObjectModel.ReadOnlyCollection`1<System.String>, Libronix.Utility.Windows.Threading.SingleInstanceManager, Libronix.Utility.Scope)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"   at LDLS4.OurApp.Main(System.String[])\n"
2024-10-17 17:59:11 DEBUG: Getting wine environment.

Did we use microsoft's icu as https://github.com/FaithLife-Community/LogosLinuxInstaller/issues/22#issuecomment-2155510687 suggests or unicode's?

n8marti commented 1 month ago

I think we need to get @tbleher into this conversation.

ctrlaltf24 commented 1 month ago

A victory! Figured out how to compile the icu.dll as windows does! (WIP branch here: https://github.com/ctrlaltf24/microsoft-icu/commit/b3368b7ffb1484cd63c22559a9091fea6a177713 ) (no longer blocked)

Still needed:

[ ] icuuc.dll and icuin.dll pointing to icu.dll (no idea how they did this)
[ ] Remove CPP exports from icu.dll (presently this code exports them, while MS Win11's doesn't) (technically works without this, but would be cleaner to remove them. There is a pre-processor flag to remove exporting them, haven't quite figured out where to put it yet)
[X] Right now the ICU data is loaded from a .dll as built from msbuilt, but we want it to pull from a .dat, we may need a compiler flag for this. Looks like the dll provided by windows neither has a dependency on the icudt.dll nor does it export the symbol. Fixed in https://github.com/ctrlaltf24/microsoft-icu/commit/be13a055844748ec31d8062c310a19e7b53e9476
[ ] Start debugging and fixing this bug!

Finally tested removing en_US_POSIX from the icudt.dat, and interestingly enough some other locale failed...continuing to debug...

n8marti commented 1 month ago

I'm trying to understand the bigger picture here. Maybe someone can help me out. We added separate ICU files because of #22. But now we've discovered that our added ICU files include (and make use of) en-us-posix language/culture, which is unrecognized by .NET shipped with Logos. Am I summarizing correctly?

If so, could it be that we "overshot" somehow and used a version of ICU that is too new for Logos? I see that Windows' current version is 72.1.0.3, while we're using 75.1. Maybe if we went went back to 72.1 it would be in sync better with Logos?

ctrlaltf24 commented 1 month ago

I'm trying to understand the bigger picture here. Maybe someone can help me out. We added separate ICU files because of #22.

Correct (these files are bundled in newer versions of windows

But now we've discovered that our added ICU files include (and make use of) en-us-posix language/culture, which is unrecognized by .NET shipped with Logos. Am I summarizing correctly?

Almost, I'm not sure what's causing this error specifically. It should be noted upstream windows does package en-us-posix in their icu data. I'm not sure what the difference is between us and windows. For now, I'm swapping the icu dlls in development to assist in the diagnosis of this bug.

If so, could it be that we "overshot" somehow and used a version of ICU that is too new for Logos? I see that Windows' current version is 72.1.0.3, while we're using 75.1. Maybe if we went went back to 72.1 it would be in sync better with Logos?

While that is possible, I'm compiling against window's 72 and still getting errors.

Progress...

Intelligence gathering

Inspected the icudtl.dat provided as a part of windows, it does ship with EN_US_POSIX, bgc and bgc_IN

Tests ran

made uloc_countAvailable return 0 to trick it into thinking there are no installed locales -> .NET exception, couldn't handle empty list
made uloc_countAvailable return 1 -> PASS! "worship" loaded successfully! This isn't a real fix, but does give us hope we can fix it on our end.
made uloc_countAvailable return 66 (right before bgc is loaded) -> PASS! "worship" loaded successfully! This isn't a real fix.
made uloc_countAvailable return 67 (including bgc but not bgc_IN) -> FAIL. Earlier hypothesis may be wrong
compiled microsoft's icu v72 got "bgc is an invalid culture identifier.\r\n" when trying to load worship.
compiled microsoft's icu v72, used clean windows 11 dat file. still got got "bgc is an invalid culture identifier.\r\n"
removed bgc from the dat file, it fails on the next one bho, what's going on? Is it somehow stopping at 64 prefixes?
removed all locales from the res_index besides en_US -> PASS. This isn't a real fix

ran icupkg against the .dat and noticed it's missing some items. We might (as a workaround) be able to remove en-us-posix from our dat file, although there still would be an underlying issue that remains unknown

$  ./icupkg.exe FaithLife/icudtl.dat
Item lang/ca_ES_VALENCIA.res depends on missing item lang/ca_ES.res
Item region/en_150.res depends on missing item region/en_001.res
Item region/en_AG.res depends on missing item region/en_001.res
Item region/en_AI.res depends on missing item region/en_001.res
Item region/en_AU.res depends on missing item region/en_001.res
...

n8marti commented 1 month ago

$ find ~/LogosBible10/data/wine64_bottle/ -iname 'icu*.d*'
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/globalization/ICU/icudtl.dat
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icu.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icuuc.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icuin.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icu.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icuuc.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icuin.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icuin52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icudt52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icuuc52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icudtl.dat

It looks like Logos ships its own icudtl.dat file. So I assume that takes precedence over anything we put into C:\windows\{system32,syswow64}. But what does that mean in terms of the #22 "windows is too old" issue? Do our new ICU files allow that test to pass, but then aren't actually used by Logos? This seems weird to me. But maybe this info will give you some :bulb:?

n8marti commented 1 month ago

$  ./icupkg.exe FaithLife/icudtl.dat
Item lang/ca_ES_VALENCIA.res depends on missing item lang/ca_ES.res
Item region/en_150.res depends on missing item region/en_001.res
Item region/en_AG.res depends on missing item region/en_001.res
Item region/en_AI.res depends on missing item region/en_001.res
Item region/en_AU.res depends on missing item region/en_001.res
...

I'm sure I'm showing my ignorance here, but I'm still trying to understand all the relevant details.

I see that en_001 is "English (World)" locale, which is different from en_US_POSIX. Not sure how important that is, but something I learned (en_001, en_US_POSIX)
Could FaithLife's icudtl.dat file simply rely on automatic fallback to the Windows icudtl.dat file, which presumably does include en_001.res?
I've tried running Logos with multiple DLL override variations, trying to isolate the problem more, but with no luck. If I force "built-in" DLLs for icu, icuin, and icuuc Logos won't start. Likewise if I block Logos' native icuin52 and icuuc52 DLLs.

ctrlaltf24 commented 1 month ago

I've tried running Logos with multiple DLL override variations, trying to isolate the problem more, but with no luck. If I force "built-in" DLLs for icu, icuin, and icuuc Logos won't start. Likewise if I block Logos' native icuin52 and icuuc52 DLLs.

Not surprising, looks like the app data versions are the same binaries that upstream normally ships, the icu.dll is actually a combination of icuin and icuuc implementations in one dll (then the icuin and icuuc dll that reside next to the icu.dll are just pointers to icu.dll)

It looks like Logos ships its own icudtl.dat file. So I assume that takes precedence over anything we put into C:\windows{system32,syswow64}. But what does that mean in terms of the https://github.com/FaithLife-Community/LogosLinuxInstaller/issues/22 "windows is too old" issue? Do our new ICU files allow that test to pass, but then aren't actually used by Logos? This seems weird to me.

Shot in the dark guess: Logos itself uses appdata, but some of logos' dependencies (perhaps the .net runtime) rely on the system icu.dll . I as was inspecting the calls I noticed getting locale information was rarely used across the logos application - when I suspect it would have been used more widely due to the number of languages shown at the same time. This is backed up by various observations when I mess with the respective dlls different components fail.

Tests Ran

ran icupkg -l against the icudt.dll in appdata and found it does include the EN_US_POSIX locale
moved the icudt72l.dat in appdata to icudt72l.dat_ - to figure out if it was used, and it was
replaced the icudt72l.dat in appdata with one that only has the en_us locale and got some weird crashes
replaced the icudt72l.dat in appdata with the one we ship into C:\Windows\Globalization\ICU/
removed the appdata dlls 2024-10-20 22:15:33 INFO: 0110:err:module:import_dll Library icuin52.dll (which is needed by L"C:\\users\\user\\AppData\\Local\\Logos\\System\\Libronix.DigitalLibrary.Resources.Native.dll") not found
tried using logos v37 - same issue

Next steps

Further investigation toward root cause

Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.

Collaboration with wine folks?

It should be noted upstream wine may want to ship these dlls in future releases of wine (as windows now natively ships them)

Workaround for Logos (and possibly wine)

@tbleher / @thw26 would it be possible to compile a new .dat using the same build environment as https://github.com/FaithLife-Community/icu/releases/tag/75.1 with the en_US_POSIX locale filtered out?

Filtering can be done using the ICU Data Build Tool. Execution example inspired by docs

# Script assumes an in-source tree build, and runConfigureICU has already been ran. Feel free to adopt steps, meant as a demonstration, not prescriptive. 
# Conditional statement to handle either microsoft's fork (icu source lives in a subdir) or upstream unicode (code resides at the root)
if [ -d icu ]; then cd icu; fi
cd icu4c/source/python
cat << EOF > filter.json
{
    "localeFilter": {
      "filterType": "regex",
      "excludelist": [
        "^en_US_POSIX$"
      ]
    }
}
EOF
python3 -m icutools.databuilder --filter=filter.json --mode=gnumake --src_dir=../data > ../data/rules.mk
rm filter.json
cd ../data/
make clean all
cp out/icudt72l.dat icudtl.dat

n8marti commented 1 month ago

Further investigation toward root cause

Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.

My next theory (maybe I'll think of a way to test it): Logos' own System.Globalization.dll is flawed (outdated?), b/c it fails to handle the valid (& properly-defined) en-us-posix culture.

Logos ships it's own .NET files (@thw26 can confirm, but I think this is partly to make things easier on us, not having to worry about installing .NET separately into wine?). The wine crash report says that Logos is using .NET 8.0.7.
I assume System.Globalization.dll is part of .NET, so maybe an update to Logos' .NET (or at least to System.Globalization.dll) will enable it to work properly. (8.0.10 binaries, 8.0.10 source code)
This isn't an issue in Windows because presumably the "default" culture is defined as something other than en-us-posix.

Even if this theory is correct, though, we obviously can't rely on tweaking Logo's installed files. So maybe there's a way instead to set the "default" culture to something other than en-us-posix. Even better if we could figure out what Windows does and match that. I searched the registry for "lang", "en_", "en-", "posix", and I didn't find an explicit entry for "en-us-posix", and I don't think Windows uses env variables at all for language settings, so I don't know where and how this "default" culture gets defined or how to override it.

Your idea of paying closer attention to .NET code (or just the system calls) might help figure out how that culture designation gets fed to Logos, though, which might help us figure out how to override it.

n8marti commented 1 month ago

Some info about how locale is determined in .NET:

dotnet/runtime/98818
.NET docs re: how culture is determined
maybe some relevant code from the unicode-org source?

thw26 commented 1 month ago

Not sure if this link has any helpful information.

https://serverfault.com/questions/559109/how-to-change-default-region-culture-settings-in-windows-2012

@ctrlaltf24, we are more than happy to ship our own ICU files. That's partially why we have our own repo, which is so that we can test for exactly these kinds of issues and distribute fixes as we need them.

I don't think it is necessarily for our benefit, but Logos does ship its own .NET files likely to have tighter control over the application's environment.

As regards Wine shipping ICU files, it may be the case that they would desire to ship them. More likely, we believe that the Winetricks folks would be the ones to ship these. We reasoned back during our work on supporting Logos 30+ that we should supply a PR to them as soon as we are able to support installing the ICU data files. Of course, if we find we need a custom version of them, that doesn't necessarily benefit us or simplify our application. Our intent was eventually to contribute ICU installation to Winetricks though.

Apologies if I missed something. I have tried to read as much as I can. Please let me know if I can answer anything else.

ctrlaltf24 commented 1 month ago

Good news, got it failing outside of Logos! And using that found the list of erroring locacles and removed them

Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.

Ran the following .NET application in the Logo's system dir (to use all of it's dependencies) /home/user/LogosBible10/data/bin/wine-devel_9.19-x86_64.AppImage /home/user/LogosBible10/data/wine64_bottle/drive_c/users/user/AppData/Local/Logos/System/dotnet-poc-en-us-posix-crash.exe

var cultures = System.Globalization.CultureInfo.GetCultures(System.Globalization.CultureTypes.AllCultures);

foreach (var culture in cultures)
{
    Console.WriteLine(culture.Name + "\t" + culture.DisplayName);
}

Unhandled exception. System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')
en-us-posix is an invalid culture identifier.
   at System.Globalization.CultureInfo.GetCultureInfo(String name)
   at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)
   at Program.<Main>$(String[] args)
0024:err:eventlog:ReportEventW L"Application: dotnet-poc-en-us-posix-crash.exe\n"
0024:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.724.31311\n"
0024:err:eventlog:ReportEventW L".NET Version: 8.0.7\n"
0024:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.\n"
0024:err:eventlog:ReportEventW L"Exception Info: System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
0024:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
0024:err:eventlog:ReportEventW L"   at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
0024:err:eventlog:ReportEventW L"   at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
0024:err:eventlog:ReportEventW L"   at Program.<Main>$(String[] args)\n"

Now we can reproduce in a lot more isolated environment!

Still no root cause, but was able to isolate and remove all the offending locales! Why these fail, I don't know. But with them removed, the problem goes away! Will work with devs on next steps

{
    "localeFilter": {
      "filterType": "regex",
      "excludelist": [
        "^bgc$",
        "^bgc_IN$",
        "^bho$",
        "^bho_IN$",
        "^cv$",
        "^cv_RU$",
        "^en_MV$",
        "^en_US_POSIX$",
        "^hi_Latn$",
        "^hi_Latn_IN$",
        "^ibb$",
        "^ibb_NG$",
        "^iu_CA$",
        "^iu_Latn_CA$",
        "^jv_Java$",
        "^jv_Java_ID$",
        "^oc_ES$",
        "^pap_029$",
        "^quc_GT$",
        "^raj$",
        "^raj_IN$",
        "^tzm_Arab$",
        "^tzm_Arab_MA$",
        "^tzm_Tfng$",
        "^tzm_Tfng_MA$"
      ]
    }
}

n8marti commented 1 month ago

Awesome! :detective: So, to summarize, the working theory is that these locales, which are defined in system-wide ICU files, are not recognized as valid by Logos' included .NET System.Globalization code. Logos requires the system-wide ICU files in order to run, but then it apparently doesn't make use of the actual definitions therein.

So does it instead only accept locales defined in its own ICU files? Or is it more complicated than that?

ctrlaltf24 commented 1 month ago

Awesome! :detective:

So, to summarize, the working theory is that these locales, which are defined in system-wide ICU files, are not recognized as valid by Logos' included .NET System.Globalization code. Logos requires the system-wide ICU files in order to run, but then it apparently doesn't make use of the actual definitions therein.

As far as I understand it yes. For some reason .NET doesn't see these locales as valid

So does it instead only accept locales defined in its own ICU files? Or is it more complicated than that?

More complicated probably, more likely windows doesn't see these locales as valid. The error happens with just the .NET runtime with Logo's shipped dlls. Probably not related to the ICU that it ships

ctrlaltf24 commented 1 month ago

Created a release on my fork with the environment and notes I used to build along with the binaries https://github.com/ctrlaltf24/microsoft-icu/releases/tag/v72.1.0.1-ctrlaltf24.1.

Feel free to test

How to test:

Download https://github.com/ctrlaltf24/microsoft-icu/releases/download/v72.1.0.1-ctrlaltf24.1/icu-win.tar.gz
(feel free to inspect the contents)
Extract the contents of that tarball into the virtual C drive
- Navigate to your Logos install directory on Linux (for example on my system I would do: cd ~/LogosBible10)
- Navigate into the virtual C:\ drive (I would do cd data/wine64_bottle/drive_c)
- Now extract the tarball here (I would do tar -xvf ~/Downloads/icu-win.tar.gz)
Open Logos as you normally do
Open a bible word study for "worship" and worship! It works!
Continue using Logos normally, lmk if there are any more crashes related to System.Globalization.CultureInfo

n8marti commented 1 month ago

More complicated probably, more likely windows doesn't see these locales as valid. The error happens with just the .NET runtime with Logo's shipped dlls. Probably not related to the ICU that it ships

I'd be interested in hearing how your test program works if it's run outside of the Logos System directory. Does it have the same problems? Maybe it won't work at all unless .NET is installed system-wide via winetricks? Or, if you can share the exe with me, I'd be willing to experiment a bit with it

n8marti commented 1 month ago

So I figured out how to build your simple C# code as a windows app, app.exe. I then used winetricks to install dotnet8 in my Logos wineprefix, then ran app.exe from C:\users\user\Desktop. And, guess what, it has the same error:

$ WINEDLLOVERRIDES="mshtml=d;" WINEPREFIX=$PWD ~/Téléchargements/wine64 ./drive_c/users/nate/Desktop/app.exe 
Unhandled exception. System.Globalization.CultureNotFoundException: Culture is n
ot supported. (Parameter 'name')
en-us-posix is an invalid culture identifier.
   at System.Globalization.CultureInfo.GetCultureInfo(String name)
   at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)
   at Program.<Main>$(String[] args) in /srv/rsync/other/app/Program.cs:line 4
0024:err:eventlog:ReportEventW L"Application: app.exe\n"
0024:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.224.6711\n"
0024:err:eventlog:ReportEventW L".NET Version: 8.0.2\n"
0024:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.\n"
0024:err:eventlog:ReportEventW L"Exception Info: System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
0024:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
0024:err:eventlog:ReportEventW L"   at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
0024:err:eventlog:ReportEventW L"   at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
0024:err:eventlog:ReportEventW L"   at Program.<Main>$(String[] args) in /srv/rsync/other/app/Program.cs:line 4\n"

So the problem is not just with Logos' .NET 8 runtime, but also with MS' own .NET 8 runtime when run within wine. I know from testing in a VM that Windows 11 doesn't have this issue with Logos or the ICU culture identifiers. But somehow the combination of wine + .NET + ICU files has this problem.

ctrlaltf24 commented 1 month ago

Existing users: you may need to install ICU files again, it'll automatically update

How to apply the update:

TUI

Utilities -> Install ICU files

CLI

Use flag --install-icu

GUI

App actions -> Install/update ICU files
Run Action

FaithLife-Community / LogosLinuxInstaller

Logos 10 crashes after opening biblical word study for "worship" #191

Found:

Open questions:

Failed attempts at producing a different result:

Observations:

unknown source

found correlation between:

Two zip differences (between upstream icu and ours):

Progress...

Intelligence gathering

Tests ran

Tests Ran

Next steps

Further investigation toward root cause

Collaboration with wine folks?

Workaround for Logos (and possibly wine)

Further investigation toward root cause

How to apply the update:

TUI

CLI

GUI