search

NightOwl888 / ICU4N

International Components for Unicode for .NET
Apache License 2.0
27 stars 7 forks source link

Problem with Collator.getInstance(), related to ICU4N.resources #69

Closed michaelhkay closed 1 month ago

michaelhkay commented 6 months ago

I'm getting an exception with Collator.getInstance(). I'm running in Rider. My project has a nuget dependency on ICU4N 60.1.0-alpha.402. Note the reference to ICU4N.resources version=60.0.0.0.

I thought I previously had this working but I might have been mistaken, because the application has a fallback path that catches the exception.

(Note, it would be great to get an ICU4N version that isn't at alpha status, since this leads to build warnings).

Michael Kay

System.TypeInitializationException: The type initializer for 'ICU4N.Globalization.CultureInfoExtensions' threw an exception. ---> System.TypeInitializationException: The type initializer for 'DotNetLocaleHelper' threw an exception. ---> System.TypeInitializationException: The type initializer for 'ICU4N.Impl.ICUData' threw an exception. ---> System.IO.FileNotFoundException: Could not load file or assembly 'ICU4N.resources, Version=60.0.0.0, Culture=neutral, PublicKeyToken=efb17c8e4f0e291b'. The system cannot find the file specified.

File name: 'ICU4N.resources, Version=60.0.0.0, Culture=neutral, PublicKeyToken=efb17c8e4f0e291b' at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound) at System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(CultureInfo culture, Version version, Boolean throwOnFileNotFound) at System.Reflection.RuntimeAssembly.GetSatelliteAssembly(CultureInfo culture, Version version) at ICU4N.Impl.ICUData..cctor() --- End of inner exception stack trace --- at ICU4N.Impl.ICUData.GetLocaleIDFromResourceName(String resourceName) at ICU4N.Impl.ICUData.GetStream(Assembly loader, String resourceName, Boolean required) at ICU4N.Impl.ICUBinary.GetData(Assembly assembly, String resourceName, String itemPath, Boolean required) at ICU4N.Impl.ICUBinary.GetData(Assembly assembly, String resourceName, String itemPath) at ICU4N.Impl.ICUResourceBundleReader.<>cDisplayClass35_0.b0(ReaderCacheKey key) at ICU4N.Impl.SoftCache`2.<>cDisplayClass1_1.b1() at System.Lazy1.ViaFactory(LazyThreadSafetyMode mode) at System.Lazy1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor) at System.Lazy1.CreateValue() at ICU4N.Impl.SoftCache2.GetOrCreate(TKey key, Func2 valueFactory) at ICU4N.Impl.ICUResourceBundleReader.GetReader(String baseName, String localeID, Assembly root) at ICU4N.Impl.ICUResourceBundle.CreateBundle(String baseName, String localeID, Assembly root) at ICU4N.Impl.ICUResourceBundle.<>c__DisplayClass64_0.<InstantiateBundle>b__0(String key) at ICU4N.Impl.SoftCache2.<>cDisplayClass1_1.b1() at System.Lazy1.ViaFactory(LazyThreadSafetyMode mode) at System.Lazy1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor) at System.Lazy1.CreateValue() at ICU4N.Impl.SoftCache2.GetOrCreate(TKey key, Func`2 valueFactory) at ICU4N.Impl.ICUResourceBundle.InstantiateBundle(String baseName, String localeID, String defaultID, Assembly root, OpenType openType) at ICU4N.Impl.ICUResourceBundle.GetBundleInstance(String baseName, String localeID, String defaultID, Assembly root, OpenType openType) at ICU4N.Impl.ICUResourceBundle.GetBundleInstance(String baseName, String localeID, Assembly root, OpenType openType) at ICU4N.Impl.ICUResourceBundle.GetBundleInstance(String baseName, String localeID, Assembly root, Boolean disableFallback) at ICU4N.Util.UResourceBundle.<>cDisplayClass22_0.b0(String key) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at ICU4N.Util.UResourceBundle.GetRootType(String baseName, Assembly root) at ICU4N.Util.UResourceBundle.InstantiateBundle(String baseName, String localeName, Assembly root, Boolean disableFallback) at ICU4N.Impl.ICUResourceBundle.CreateUCultureList(String baseName, Assembly root) at ICU4N.Impl.ICUResourceBundle.AvailEntry.<>c__DisplayClass7_0.b__0() at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Func1 valueFactory) at ICU4N.Impl.ICUResourceBundle.AvailEntry.GetUCultureList(UCultureTypes types) at ICU4N.Impl.ICUResourceBundle.GetUCultures(String baseName, Assembly assembly, UCultureTypes types) at ICU4N.Impl.ICUResourceBundle.GetUCultures(UCultureTypes types) at ICU4N.Globalization.UCultureInfo.GetCultures(UCultureTypes types) at ICU4N.Globalization.UCultureInfo.DotNetLocaleHelper.LoadNonGregorianDefaultCalendars() at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func1 valueFactory) at ICU4N.Globalization.UCultureInfo.DotNetLocaleHelper.EnsureInitialized() at ICU4N.Globalization.UCultureInfo.DotNetLocaleHelper..cctor() --- End of inner exception stack trace --- at ICU4N.Globalization.UCultureInfo.DotNetLocaleHelper.EnsureInitialized() at ICU4N.Globalization.CultureInfoExtensions..cctor() --- End of inner exception stack trace --- at ICU4N.Globalization.CultureInfoExtensions.ToUCultureInfo(CultureInfo culture) at ICU4N.Globalization.UCultureInfo.GetCurrentCulture() at ICU4N.Globalization.UCultureInfo.get_CurrentCulture() at ICU4N.Text.Collator.GetInstance() at Saxon.Eej.expr.sort.UcaCollatorUsingIcu..ctor(String uri) in /Users/mike/GitHub/saxon13/build/cs/ee/Saxon/Eej/expr/sort/UcaCollatorUsingIcu.cs:line 22 The type initializer for 'ICU4N.Globalization.CultureInfoExtensions' threw an exception. System.TypeInitializationException: The type initializer for 'ICU4N.Globalization.CultureInfoExtensions' threw an exception. ---> System.TypeInitializationException: The type initializer for 'DotNetLocaleHelper' threw an exception. ---> System.TypeInitializationException: The type initializer for 'ICU4N.Impl.ICUData' threw an exception. ---> System.IO.FileNotFoundException: Could not load file or assembly 'ICU4N.resources, Version=60.0.0.0, Culture=neutral, PublicKeyToken=efb17c8e4f0e291b'. The system cannot find the file specified.

NightOwl888 commented 6 months ago

Yes, you are correct that it should fallback. However, there are 2 exceptions:

  1. If the neutral culture (e.g. en/ICU4N.resources.dll) is not present but a localized version of it (e.g. en-GB/ICU4N.resources.dll) is present, it will fail because most of the data is in the neutral culture assembly. This is a limitation imposed in ICU4J.
  2. If the root assembly ICU4N.resources.dll is missing, ICU4N will fail to execute since this is a required file.

I suspect it is more likely the latter situation that is causing this error. But you will need to check your bin folder to see what is actually on disk.

If you only have a PackageReference to ICU4N and have not customized the build in any way, it should copy the files in appropriately. But a good test to see if the build is working is to close the IDE, delete the /bin and /obj folders, and then build again.

I think we will have to take a step backward in the next alpha release and put all of the resources in a single root ICU4N.resources.dll satellite assembly file, as getting into the weeds of which culture names are appropriate to use turns out to be more nuanced than "gather the list of cultures". Microsoft has added cultures that ICU4C doesn't support out of the box, which makes the decision to follow Microsoft's approach or use the standard ICU resources a tough choice. Not to mention, the appropriate list may vary depending on the .NET target framework. It doesn't seem worth it to delay the next release while arriving at a decision on this, but we definitely need a better approach before then.

I have been putting in several 16-hour days over the past few weeks adding wide support for ReadOnlySpan<char> so we can eliminate the need to generate code with multiple copies of the same business logic in it and maintain APIs that accept char[], StringBuilder, and ICharSequence. This work is complete in J2N and around 2/3 of the way complete in ICU4N. This of course goes a long way toward making the public API more stable.

But ultimately, I think much of the public API surface needs to be made internal (or flagged in a way that the user receives warnings if they use it) so we can do a production release without getting into the weeds of refactoring either because of conventions or performance degradation due to unfortunate API design choices.

michaelhkay commented 6 months ago

Thanks for the response. I can confirm that there were no ICU4N.resources files in the bin folder. I've deleted the bin and obj folders as suggested, and the immediate effect is an error Microsoft.PackageDependencyResolution.targets(266, 5): [NETSDK1004] Assets file '/Users/mike/RiderProjects/.../obj/project.assets.json' not found. Run a NuGet package restore to generate this file.

I'm attempting to restore the project, with mixed success. I think I'll start rebuilding it from scratch and see what happens. At least I now have some pointers.

michaelhkay commented 6 months ago

I've now rebuilt everything: did a good clear-out of my .NET and Rider installations, and created a new solution/project.

I'm getting the same failure.

A file named ICU4N.resources.dll is present in all of:

I'm not sure what you mean by "If the root assembly ICU4N.resources.dll is missing" - presumably the first of those three. (Which also contains ICU4N.dll and othe dependencies).

michaelhkay commented 6 months ago

I have now established that the same basic code is working in another project (with an earlier version of Saxon). The most obvious difference between the projects is that the failing one is .net8.0, the working one is .net6.0. But switching the failing project to .net6.0 makes no difference. Any suggestions where I start looking?

NightOwl888 commented 6 months ago

I took a look by setting up a new .NET 8 project and noticed some strange behavior. I put 2 projects into a solution, a class library with a package reference on ICU4N and a console app. The resource files get copied into the console app output, but not in the class library.

I then switched the projects to net6.0 and got a similar result. However, I noticed that there is a difference. In the class library that targets net6.0, the <project>.deps.json file doesn't contain any of the resource files, but the file with the same name in the nt8.0 folder lists all of them.

That said, since a class library project is not executable, I don't think this is an issue (at least not in a production scenario). The resources get copied into a console app, so they exist in the place where the code runs. I also added a test project and confirmed that they get copied into that project and the tests run in the IDE.

Then I tried running the test project using dotnet test -c Debug, and reproduced the exception. I tried the test again in Visual Studio 2022 and it errored there, as well. There was a yellow icon next to the ICU4N.Resources package reference in the IDE. So, I then attempted to explicitly add a package reference to ICU4N.Resources from the test project and it now works on both the command line and in the IDE.

So, in summary, I have confirmed this is a bug, but at this point I don't know whether it is something that was introduced with the .NET 8 SDK or it is something that is broken on our end. You should be able to continue working by adding an explicit reference to ICU4N.Resources in your project until a fix is available.

michaelhkay commented 6 months ago

Thanks, that seems to have solved it for now.

Dynamic loading in .NET is a pretty delicate business. We're pretty well eliminated it in SaxonCS because it was causing us too much support hassle.