search

dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.73k stars 1.07k forks source link

donnet 3.1 seems broken on last openSUSE Tumbleweed #25069

Closed Gankov closed 2 years ago

Gankov commented 2 years ago

Describe the bug

After last OS update dotnet broken. It always show error:

Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

Perhaps, it cause icu was update from 70 version to 71 version. Maybe this is bug in icu package of openSUSE, but i don't know why dotnet cant't find icu, and how check that icu is broken. In OS, icu packages are installed. When i load to snapshot before update icu to 71, dotnet works.

To Reproduce

1 - install dotnet-sdk on last openSUSE Tumbleweed 2 - try use any dotnet commands

Exceptions (if any)

Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support. at System.Environment.FailFast(System.String) at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode() at System.Globalization.GlobalizationMode..cctor() at System.Globalization.CultureData.CreateCultureWithInvariantData() at System.Globalization.CultureData.get_Invariant() at System.Globalization.CultureInfo..cctor() at System.String.ToUpperInvariant() at System.Diagnostics.Tracing.EventSource.GetGuid(System.Type) at System.Diagnostics.Tracing.EventSource..ctor(System.Diagnostics.Tracing.EventSourceSettings, System.String[]) at System.Diagnostics.Tracing.EventSource..ctor() at Microsoft.Build.Eventing.MSBuildEventSource..cctor() at Microsoft.Build.CommandLine.MSBuildApp.Execute(System.String[]) at Microsoft.Build.CommandLine.MSBuildApp.Main(System.String[])

Further technical details

dotnet-issue-labeler[bot] commented 2 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

am11 commented 2 years ago

Thank you for the report.

@janvorli, @tarekgh, some distros have upgraded their ICU package. Would it be feasible to backport this change https://github.com/dotnet/runtime/commit/829baa4dcfe8c32034ebb4cf867dff07d853c04a to 3.1 (https://github.com/dotnet/coreclr/blob/master/src/corefx/System.Globalization.Native/pal_icushim.c#L60)? 3.1 is supported until December 3, 2022.

tarekgh commented 2 years ago

@Gankov looking at the page https://software.opensuse.org/package/icu I am seeing openSUSE is using ICU version 65. Did you install version 71 manually? or is this version installed automatically on your openSUSE? Also, is it possible for you to migrate to .NET 6.0? As noted, 3.1 is not going to be supported by the end of the current year.

.NET Core 3.1 has limitation to the max ICU version it can use which is 70.

tarekgh commented 2 years ago

@am11 we may consider the fix porting if the openSUSE is using ICU version 71 by default or it is a blocker issue. Otherwise, I would say using ICU version 70 would be acceptable. Note, Windows is still using ICU version 68 which is acceptable so far.

am11 commented 2 years ago

It looks like the ICU 7.1 package is also available in their repository https://software.opensuse.org/package/libicu71. It might be that package manager in @Gankov's machine found the latest available version (rather than the unsuffixed version 65)?

Gankov commented 2 years ago

@Gankov looking at the page https://software.opensuse.org/package/icu I am seeing openSUSE is using ICU version 65. Did you install version 71 manually? or is this version installed automatically on your openSUSE?

I use openSUSE Tumbleweed. This is rolling distro with last packages. On your link can see that Tumbleweed have 71 version. Yes, it is default version for Tumbleweed now. But openSUSE Leap 15.3 have 65 version of icu.

Also, is it possible for you to migrate to .NET 6.0? As noted, 3.1 is not going to be supported by the end of the current year. .NET Core 3.1 has limitation to the max ICU version it can use which is 70.

I will can try NET6, I use Core 3.1 because i tred use NET5 year ago, but after installed NET5 broked build Net Framework projects on linux, we can't full migrate to dotnet yet, because we use some legacy code, we work it.

tarekgh commented 2 years ago

I use openSUSE Tumbleweed. This is rolling distro with last packages. On your link can see that Tumbleweed have 71 version.

Thanks for the info. Is it possible to use a different ICU version there? I mean downgrade the ICU version. This should work around the issue you are running into.

I will can try NET6, I use Core 3.1 because i tred use NET5 year ago, but after installed NET5 broked build Net Framework projects on linux, we can't full migrate to dotnet yet, because we use some legacy code, we work it.

I would suggest start the migration to .NET 6.0 as we are getting closer to the retirement date of 3.1.

Gankov commented 2 years ago

Thanks for the info. Is it possible to use a different ICU version there? I mean downgrade the ICU version. This should work around the issue you are running into.

For openSUSE Tumbleweed have not simple way to downgrade the ICU version. I tried it, but now there is only icu 71 in official repository. I tried install from three party repository, but some system packages have dependence on 71. :(

janvorli commented 2 years ago

@Gankov would setting the CLR_ICU_VERSION_OVERRIDE env variable to the version of ICU you are using before launching your app be an acceptable workaround? The format of the string in this variable is majorVer[.minorVer[.subVer]] (the brackets indicate optional parts).

Gankov commented 2 years ago

@Gankov would setting the CLR_ICU_VERSION_OVERRIDE env variable to the version of ICU you are using before launching your app be an acceptable workaround? The format of the string in this variable is majorVer[.minorVer[.subVer]] (the brackets indicate optional parts).

Yes, it works for me. Thanks.

tarekgh commented 2 years ago

Closing the issue as the workaround worked fine.

tonibgd commented 2 years ago

@tarekgh

I have both 3.1 and 6.0 installed. Building and running projects that target net6.0 fail with icu 71.1:

Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

Using the workaround CLR_ICU_VERSION_OVERRIDE=71.1 works.

Runtime Environment: OS Name: manjaro OS Version:
OS Platform: Linux RID: linux-x64 Base Path: /usr/share/dotnet/sdk/6.0.301/

Host (useful for support): Version: 6.0.6 Commit: 7cca709db2

.NET SDKs installed: 3.1.414 [/usr/share/dotnet/sdk] 6.0.301 [/usr/share/dotnet/sdk]

.NET runtimes installed: Microsoft.AspNetCore.App 3.1.20 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 3.1.20 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs: https://aka.ms/dotnet-download


- `pacman -Si icu`

Repository : core Name : icu Version : 71.1-1 Description : International Components for Unicode library Architecture : x86_64 URL : https://icu.unicode.org Licenses : custom:icu Groups : None Provides : libicudata.so=71-64 libicui18n.so=71-64 libicuio.so=71-64 libicutest.so=71-64 libicutu.so=71-64 libicuuc.so=71-64 Depends On : gcc-libs sh Optional Deps : None Conflicts With : None Replaces : None Download Size : 10.85 MiB Installed Size : 39.73 MiB Packager : Andreas Radke andyrtr@archlinux.org Build Date : Thu 14 Apr 2022 04:09:12 PM CEST Validated By : MD5 Sum SHA-256 Sum Signature

tarekgh commented 2 years ago

@tonibgd I have installed OpenSUSE tumbleweed on WSL and installed .NET 6.0 using the instructions mentioned in https://docs.microsoft.com/en-us/dotnet/core/install/linux-opensuse. Then I created a simple console application, and it runs fine. 

OSDescription: Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022
FrameworkDescription: .NET 6.0.6
RuntimeIdentifier: opensuse-tumbleweed.20220415-x64
6.0.6
.... /usr/lib64/libicui18n.so.71.1:
.... /usr/lib64/libicuuc.so.71.1:
.... ICU Version: 71.1.0.0
.... UseNls:                 False ....
.... Invariant:              False ....
.... PredefinedCulturesOnly: False ....

here is the code I used to dump these values:

            Console.WriteLine($"OSDescription: {RuntimeInformation.OSDescription}");
            Console.WriteLine($"FrameworkDescription: {RuntimeInformation.FrameworkDescription}");
#if NET5_0_OR_GREATER
            Console.WriteLine($"RuntimeIdentifier: {RuntimeInformation.RuntimeIdentifier}");
#endif // NET5_0_OR_GREATER
            Console.WriteLine($"{Environment.Version}");

            try
            {
                foreach (ProcessModule module in Process.GetCurrentProcess().Modules)
                {
                    if (module.FileName!.IndexOf("icu", StringComparison.OrdinalIgnoreCase) >= 0)
                        Console.WriteLine($".... {module.FileName}: {module.FileVersionInfo.FileVersion}");
                }

                Type? interopGlobalization = Type.GetType("Interop+Globalization, System.Private.CoreLib");
                if (interopGlobalization != null)
                {
                    MethodInfo? methodInfo = interopGlobalization.GetMethod("GetICUVersion", BindingFlags.NonPublic | BindingFlags.Static);
                    if (methodInfo != null)
                    {
                        int version = (int)methodInfo.Invoke(null, null)!;
                        Console.WriteLine($".... ICU Version: {new Version(version >> 24, (version >> 16) & 0xFF, (version >> 8) & 0xFF, version & 0xFF)}");
                    }
                }
                Console.WriteLine($".... UseNls:                 {typeof(object).Assembly.GetType("System.Globalization.GlobalizationMode")!.GetProperty("UseNls", BindingFlags.Static | BindingFlags.NonPublic)!.GetValue(null)} ....");
                Console.WriteLine($".... Invariant:              {typeof(object).Assembly.GetType("System.Globalization.GlobalizationMode")!.GetProperty("Invariant", BindingFlags.Static | BindingFlags.NonPublic)!.GetValue(null)} ....");
                Console.WriteLine($".... PredefinedCulturesOnly: {typeof(object).Assembly.GetType("System.Globalization.GlobalizationMode")!.GetProperty("PredefinedCulturesOnly", BindingFlags.Static | BindingFlags.NonPublic)!.GetValue(null)} ....");
            }
            catch
            {
                Console.WriteLine($".... UseNls: Couldn't Evaluate it ....");
            }

The only difference I guess is I am using 6.0.6 but I don't think this can cause the loading ICU work. Looks to me there is something wrong somehow in your environment.

tonibgd commented 2 years ago

@tarekgh Thanks for looking into it, I don't have access to OpenSUSE at the moment, i'm currently on Manjaro (Arch based) but both of them have the latest ICU.

I think I narrowed it down to Microsoft.NET.Sdk.Functions package:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net6.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="4.1.1" />
    </ItemGroup>
</Project>
tarekgh commented 2 years ago

I think I narrowed it down to Microsoft.NET.Sdk.Functions package

What do you mean by that? Do you mean referencing this package causing the problem? I don't think we officially support .NET on Manjaro.

tonibgd commented 2 years ago

Yes, referencing the package causes the build to fail, here is the full stack trace:

  Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator.targets(37, 5): Metadata generation failed. Exit code: '134' Error: 'Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.   at System.Environment.FailFast(System.String)   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()   at System.Globalization.GlobalizationMode..cctor()   at System.Globalization.CultureData.CreateCultureWithInvariantData()   at System.Globalization.CultureData.get_Invariant()   at System.Globalization.CultureInfo..cctor()   at System.String.ToLowerInvariant()   at System.IO.PathInternal.GetIsCaseSensitive()   at System.IO.PathInternal..cctor()   at System.IO.Enumeration.FileSystemEnumerableFactory.MatchesPattern(System.String, System.ReadOnlySpan`1<Char>, System.IO.EnumerationOptions)   at System.IO.Enumeration.FileSystemEnumerableFactory+<>c__DisplayClass3_0.<UserFiles>b__1(System.IO.Enumeration.FileSystemEntry ByRef)   at System.IO.Enumeration.FileSystemEnumerable`1+DelegateEnumerator[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ShouldIncludeEntry(System.IO.Enumeration.FileSystemEntry ByRef)   at System.IO.Enumeration.FileSystemEnumerator`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()   at System.Linq.Enumerable+WhereEnumerableIterator`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ToArray()   at System.Linq.Enumerable.ToArray[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>)   at ExtensionsMetadataGenerator.ExtensionsMetadataGenerator.Generate(System.String, System.String, ExtensionsMetadataGenerator.ConsoleLogger)   at ExtensionsMetadataGenerator.Console.Program.Main(System.String[])'
tarekgh commented 2 years ago

I am seeing in your stack System.Globalization.GlobalizationMode.GetGlobalizationInvariantMod this method doesn't exist in .NET 6.0 anymore and exists in .NET 3.1. I believe this failure is triggered from 3.1 run.

tonibgd commented 2 years ago

Thanks for looking into it, it appears .NET Core 3.1 runtime is causing the issue and preventing .NET 6.0 builds that reference Microsoft.NET.Sdk.Functions.

I need .NET Core 3.1 runtime for both my project and also I use some other tools that depend on 3.1, for example Azure Storage Explorer.

I will be using CLR_ICU_VERSION_OVERRIDE=71.1 workaround until I can remove 3.1 runtime

Edit: The same solution has many projects, most are 6.0 but some are still on 3.1, that is what’s causing the issue.