search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.45k stars 4.76k forks source link

Strange errors in .NET SDK 7.0.105 ppc64le #86069

Closed ww898 closed 4 months ago

ww898 commented 1 year ago

Hi, my target was testing some very small .NET projects in Linux s390x / ppc64le environment. I found temporary access to IBM cloud LinuxONE s390x and succefully did my job, but I didn't find any free access to IBM Power cloud. So, I deceided to try QEMU for that. I found that there isn't ability to download tar.gz with s390x/ppc64le from official Microsoft site directly and dotnet-install.sh doen't find required archives. The only way which I found was running Linux Fedora v36-v39 s390x/ppc64le under QEMU and install .NET SDK 7.0 from package manager:

# dotnet --info
.NET SDK:
 Version:   7.0.105
 Commit:    e1bc5e001c

Runtime Environment:
 OS Name:     fedora
 OS Version:  39
 OS Platform: Linux
 RID:         fedora.39-ppc64le
 Base Path:   /usr/lib64/dotnet/sdk/7.0.105/

Host:
  Version:      7.0.5
  Architecture: ppc64le
  Commit:       8042d61b17

.NET SDKs installed:
  7.0.105 [/usr/lib64/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]

I failed to run dotnet build on my solution because of errors: /usr/lib64/dotnet/sdk/7.0.105/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(221,5): error : The Regex engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors..

I tryed to run dotnet test -f net7.0 directly and got a lot of exceptions:

System.OutOfMemoryException: Insufficient memory to continue the execution of the program.
   at System.Collections.Hashtable.rehash(Int32 newsize)
   at System.Collections.Hashtable.expand()
   at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
   at System.Collections.Hashtable.set_Item(Object key, Object value)
   at System.ComponentModel.TypeDescriptor.CheckDefaultProvider(Type type)
   at System.ComponentModel.TypeDescriptor.NodeFor(Type type, Boolean createDelegator)
   at System.ComponentModel.TypeDescriptor.NodeFor(Type type)
   at System.ComponentModel.TypeDescriptor.GetDescriptor(Type type, String typeName)
   at System.ComponentModel.TypeDescriptor.GetConverter(Type type)
   at Newtonsoft.Json.Serialization.JsonTypeReflector.CanTypeDescriptorConvertString(Type type, TypeConverter& typeConverter)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.CanConvertToString(Type type)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract(Type objectType)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.Serialization.DefaultTestPlatformContractResolver.CreateContract(Type objectType)
   at System.Collections.Concurrent.ConcurrentDictionary`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].GetOrAdd(Type key, Func`2 valueFactory)
   at Newtonsoft.Json.Utilities.ThreadSafeStore`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].Get(Type key)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract(Type type)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContract(Object value)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContractSafe(Object value)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer.SerializePayload(String messageType, Object payload, Int32 version)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.SendExecutionComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.EventHandlers.TestRunEventsHandler.HandleTestRunComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RaiseTestRunComplete(Exception exception, Boolean canceled, Boolean aborted, TimeSpan elapsedTime)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTests()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.ExecutionManager.StartTestRun(Dictionary`2 adapterSourceMap, String package, String runSettings, TestExecutionContext testExecutionContext, ITestCaseEventsHandler testCaseEventsHandler, IInternalTestRunEventsHandler runEventsHandler)

and everything hangs

I repeated all steps on s390x architecture instead of ppc64le - everything works properly. The .NET SDK version is the same - 7.0.105.

Host OS: Ubuntu 22.04.2 LTS + Docker 23.0.5 Steps to reproduce (the current directory is the dirctory with solution file):

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker run --rm -it -v "$(pwd):/work" --platform linux/ppc64le ppc64le/fedora:39
dnf install -t dotnet-skd-7.0
cd /work
dotnet restore
dotnet test -f net7.0 --no-restore

Sometimes I got following error in additional: The active test run was aborted. Reason: Error converting value "00:00:00.0771500" to type 'System.TimeSpan'. Path 'Payload.NewTestResults[0].Duration', line 1, position 783.

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis See info in area-owners.md if you want to be subscribed.

Issue Details
Hi, my target was testing some very small .NET projects in Linux s390x / ppc64le environment. I found temporary access to IBM cloud LinuxONE s390x and succefully did my job, but I didn't find any free access to IBM Power cloud. So, I deceided to try QEMU for that. I found that there isn't ability download tar.gz with s390x/ppc64le from official Microsoft site directly and `dotnet-install.sh` doen't find required archives. The only way which I found was running Linux Fedora v36-v39 s390x/ppc64le under QEMU and install .NET SDK 7.0 from package manager: ``` # dotnet --info .NET SDK: Version: 7.0.105 Commit: e1bc5e001c Runtime Environment: OS Name: fedora OS Version: 39 OS Platform: Linux RID: fedora.39-ppc64le Base Path: /usr/lib64/dotnet/sdk/7.0.105/ Host: Version: 7.0.5 Architecture: ppc64le Commit: 8042d61b17 .NET SDKs installed: 7.0.105 [/usr/lib64/dotnet/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App] ``` I failed to run `dotnet build` my solution because of errors `/usr/lib64/dotnet/sdk/7.0.105/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(221,5): error : The Regex engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors.`. I tryed to run `dotnet test -f net7.0` directly and got a lot of exceptions: ``` System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Collections.Hashtable.rehash(Int32 newsize) at System.Collections.Hashtable.expand() at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add) at System.Collections.Hashtable.set_Item(Object key, Object value) at System.ComponentModel.TypeDescriptor.CheckDefaultProvider(Type type) at System.ComponentModel.TypeDescriptor.NodeFor(Type type, Boolean createDelegator) at System.ComponentModel.TypeDescriptor.NodeFor(Type type) at System.ComponentModel.TypeDescriptor.GetDescriptor(Type type, String typeName) at System.ComponentModel.TypeDescriptor.GetConverter(Type type) at Newtonsoft.Json.Serialization.JsonTypeReflector.CanTypeDescriptorConvertString(Type type, TypeConverter& typeConverter) at Newtonsoft.Json.Serialization.DefaultContractResolver.CanConvertToString(Type type) at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract(Type objectType) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.Serialization.DefaultTestPlatformContractResolver.CreateContract(Type objectType) at System.Collections.Concurrent.ConcurrentDictionary`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].GetOrAdd(Type key, Func`2 valueFactory) at Newtonsoft.Json.Utilities.ThreadSafeStore`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].Get(Type key) at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract(Type type) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContract(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContractSafe(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer.SerializePayload(String messageType, Object payload, Int32 version) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.SendExecutionComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.EventHandlers.TestRunEventsHandler.HandleTestRunComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RaiseTestRunComplete(Exception exception, Boolean canceled, Boolean aborted, TimeSpan elapsedTime) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTests() at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.ExecutionManager.StartTestRun(Dictionary`2 adapterSourceMap, String package, String runSettings, TestExecutionContext testExecutionContext, ITestCaseEventsHandler testCaseEventsHandler, IInternalTestRunEventsHandler runEventsHandler) ``` and **everything hangs** I repeated all steps on s390x architecture instead of ppc64le - everything works properly. The .NET SDK version is the same - 7.0.105. Host OS: Ubuntu 22.04.2 LTS + Docker 23.0.5 Steps to reproduce (the current directory is the dirctory with solution file): ``` docker run --rm --privileged multiarch/qemu-user-static --reset -p yes docker run --rm -it -v "$(pwd):/work" --platform linux/ppc64le ppc64le/fedora:39 dnf install -t dotnet-skd-7.0 cd /work dotnet restore dotnet test -f net7.0 --no-restore ```
Author: ww898
Assignees: -
Labels: `area-System.Text.Json`
Milestone: -
dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

marcpopMSFT commented 1 year ago

I think there is a known issue in regex with qemu: https://github.com/dotnet/runtime/issues/78340 Moving back to runtime.

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis See info in area-owners.md if you want to be subscribed.

Issue Details
Hi, my target was testing some very small .NET projects in Linux s390x / ppc64le environment. I found temporary access to IBM cloud LinuxONE s390x and succefully did my job, but I didn't find any free access to IBM Power cloud. So, I deceided to try QEMU for that. I found that there isn't ability to download tar.gz with s390x/ppc64le from official Microsoft site directly and `dotnet-install.sh` doen't find required archives. The only way which I found was running Linux Fedora v36-v39 s390x/ppc64le under QEMU and install .NET SDK 7.0 from package manager: ``` # dotnet --info .NET SDK: Version: 7.0.105 Commit: e1bc5e001c Runtime Environment: OS Name: fedora OS Version: 39 OS Platform: Linux RID: fedora.39-ppc64le Base Path: /usr/lib64/dotnet/sdk/7.0.105/ Host: Version: 7.0.5 Architecture: ppc64le Commit: 8042d61b17 .NET SDKs installed: 7.0.105 [/usr/lib64/dotnet/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App] ``` I failed to run `dotnet build` on my solution because of errors: `/usr/lib64/dotnet/sdk/7.0.105/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(221,5): error : The Regex engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors.`. I tryed to run `dotnet test -f net7.0` directly and got a lot of exceptions: ``` System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Collections.Hashtable.rehash(Int32 newsize) at System.Collections.Hashtable.expand() at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add) at System.Collections.Hashtable.set_Item(Object key, Object value) at System.ComponentModel.TypeDescriptor.CheckDefaultProvider(Type type) at System.ComponentModel.TypeDescriptor.NodeFor(Type type, Boolean createDelegator) at System.ComponentModel.TypeDescriptor.NodeFor(Type type) at System.ComponentModel.TypeDescriptor.GetDescriptor(Type type, String typeName) at System.ComponentModel.TypeDescriptor.GetConverter(Type type) at Newtonsoft.Json.Serialization.JsonTypeReflector.CanTypeDescriptorConvertString(Type type, TypeConverter& typeConverter) at Newtonsoft.Json.Serialization.DefaultContractResolver.CanConvertToString(Type type) at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract(Type objectType) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.Serialization.DefaultTestPlatformContractResolver.CreateContract(Type objectType) at System.Collections.Concurrent.ConcurrentDictionary`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].GetOrAdd(Type key, Func`2 valueFactory) at Newtonsoft.Json.Utilities.ThreadSafeStore`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].Get(Type key) at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract(Type type) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContract(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContractSafe(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer.SerializePayload(String messageType, Object payload, Int32 version) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.SendExecutionComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.EventHandlers.TestRunEventsHandler.HandleTestRunComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RaiseTestRunComplete(Exception exception, Boolean canceled, Boolean aborted, TimeSpan elapsedTime) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTests() at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.ExecutionManager.StartTestRun(Dictionary`2 adapterSourceMap, String package, String runSettings, TestExecutionContext testExecutionContext, ITestCaseEventsHandler testCaseEventsHandler, IInternalTestRunEventsHandler runEventsHandler) ``` and **everything hangs** I repeated all steps on s390x architecture instead of ppc64le - everything works properly. The .NET SDK version is the same - 7.0.105. Host OS: Ubuntu 22.04.2 LTS + Docker 23.0.5 Steps to reproduce (the current directory is the dirctory with solution file): ``` docker run --rm --privileged multiarch/qemu-user-static --reset -p yes docker run --rm -it -v "$(pwd):/work" --platform linux/ppc64le ppc64le/fedora:39 dnf install -t dotnet-skd-7.0 cd /work dotnet restore dotnet test -f net7.0 --no-restore ``` Sometimes I got following error in additional: `The active test run was aborted. Reason: Error converting value "00:00:00.0771500" to type 'System.TimeSpan'. Path 'Payload.NewTestResults[0].Duration', line 1, position 783.`
Author: ww898
Assignees: nagilson
Labels: `area-System.Text.Json`
Milestone: -
gregsdennis commented 1 year ago

The exception highlights Newtonsoft. I don't think it has anything to do with STJ.

tarekgh commented 1 year ago

Per @AndyAyersMS comment https://github.com/dotnet/runtime/issues/78340#issuecomment-1505811840 running under QEMU is not officially supported.

CC @stephentoub

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-meta See info in area-owners.md if you want to be subscribed.

Issue Details
Hi, my target was testing some very small .NET projects in Linux s390x / ppc64le environment. I found temporary access to IBM cloud LinuxONE s390x and succefully did my job, but I didn't find any free access to IBM Power cloud. So, I deceided to try QEMU for that. I found that there isn't ability to download tar.gz with s390x/ppc64le from official Microsoft site directly and `dotnet-install.sh` doen't find required archives. The only way which I found was running Linux Fedora v36-v39 s390x/ppc64le under QEMU and install .NET SDK 7.0 from package manager: ``` # dotnet --info .NET SDK: Version: 7.0.105 Commit: e1bc5e001c Runtime Environment: OS Name: fedora OS Version: 39 OS Platform: Linux RID: fedora.39-ppc64le Base Path: /usr/lib64/dotnet/sdk/7.0.105/ Host: Version: 7.0.5 Architecture: ppc64le Commit: 8042d61b17 .NET SDKs installed: 7.0.105 [/usr/lib64/dotnet/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 7.0.5 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App] ``` I failed to run `dotnet build` on my solution because of errors: `/usr/lib64/dotnet/sdk/7.0.105/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(221,5): error : The Regex engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors.`. I tryed to run `dotnet test -f net7.0` directly and got a lot of exceptions: ``` System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Collections.Hashtable.rehash(Int32 newsize) at System.Collections.Hashtable.expand() at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add) at System.Collections.Hashtable.set_Item(Object key, Object value) at System.ComponentModel.TypeDescriptor.CheckDefaultProvider(Type type) at System.ComponentModel.TypeDescriptor.NodeFor(Type type, Boolean createDelegator) at System.ComponentModel.TypeDescriptor.NodeFor(Type type) at System.ComponentModel.TypeDescriptor.GetDescriptor(Type type, String typeName) at System.ComponentModel.TypeDescriptor.GetConverter(Type type) at Newtonsoft.Json.Serialization.JsonTypeReflector.CanTypeDescriptorConvertString(Type type, TypeConverter& typeConverter) at Newtonsoft.Json.Serialization.DefaultContractResolver.CanConvertToString(Type type) at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract(Type objectType) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.Serialization.DefaultTestPlatformContractResolver.CreateContract(Type objectType) at System.Collections.Concurrent.ConcurrentDictionary`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].GetOrAdd(Type key, Func`2 valueFactory) at Newtonsoft.Json.Utilities.ThreadSafeStore`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].Get(Type key) at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract(Type type) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContract(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.GetContractSafe(Object value) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer.SerializePayload(String messageType, Object payload, Int32 version) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.SendExecutionComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.EventHandlers.TestRunEventsHandler.HandleTestRunComplete(TestRunCompleteEventArgs testRunCompleteArgs, TestRunChangedEventArgs lastChunkArgs, ICollection`1 runContextAttachments, ICollection`1 executorUris) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RaiseTestRunComplete(Exception exception, Boolean canceled, Boolean aborted, TimeSpan elapsedTime) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTests() at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.ExecutionManager.StartTestRun(Dictionary`2 adapterSourceMap, String package, String runSettings, TestExecutionContext testExecutionContext, ITestCaseEventsHandler testCaseEventsHandler, IInternalTestRunEventsHandler runEventsHandler) ``` and **everything hangs** I repeated all steps on s390x architecture instead of ppc64le - everything works properly. The .NET SDK version is the same - 7.0.105. Host OS: Ubuntu 22.04.2 LTS + Docker 23.0.5 Steps to reproduce (the current directory is the dirctory with solution file): ``` docker run --rm --privileged multiarch/qemu-user-static --reset -p yes docker run --rm -it -v "$(pwd):/work" --platform linux/ppc64le ppc64le/fedora:39 dnf install -t dotnet-skd-7.0 cd /work dotnet restore dotnet test -f net7.0 --no-restore ``` Sometimes I got following error in additional: `The active test run was aborted. Reason: Error converting value "00:00:00.0771500" to type 'System.TimeSpan'. Path 'Payload.NewTestResults[0].Duration', line 1, position 783.`
Author: ww898
Assignees: nagilson
Labels: `area-Meta`, `untriaged`
Milestone: -
akoeplinger commented 1 year ago

@uweigand do you know if running under QEMU works for Mono on ppc64le/s390x? I remember hitting issues way back when the port started.

uweigand commented 1 year ago

@akoeplinger on s390x I think it should work assuming a recent qemu version, but I haven't done much testing. I don't know if there are any issues with ppc64le though - @Sapana-Khemkar, maybe you or Janani can follow up on this? Also, I think there should be some way to get access to Power systems for open-source work as well, can you help with this?

janani66 commented 1 year ago

Access to Power Systems for open-source work

In partnership with IBM, the Open Source Lab (OSL) provide access to IBM Power processor-based servers for developing and testing open source projects. There are two clusters available, and access is free to qualified open source developers. A variety of Linux distributions are available.

The OpenStack (non-GPU) based cluster offers Power little endian (LE) instances running on kernel-based virtual machine (KVM) with access via Secure Shell (SSH) and/or through OpenStack's API and GUI interface. This environment is one the best options for functional development and continuous integration (CI) work via the managed Jenkins service hosted on the cluster or as a node incorporated into an external continuous integration/continuous delivery (CI/CD) pipeline. Register for one or more GitHub repositories on the Jenkins server and configure the build process and the environment as needed.

Learn more Request access to OpenStack cluster Request access to Jenkins Power CI

ww898 commented 10 months ago

Hi there again,

I finally got the access to real ppc64le hardware (thanks OpenPOWER Foundation). The issue System.OutOfMemoryException : Insufficient memory to continue the execution of the program. is still here for both .NET 7.0 and 8.0. Now I can confidently say that the problem is in .NET and is simultaneously reproduced on hardware and on QEMU.

How to reproduce the issue on Fedora 39 ppc64le:

sudo yum update -y
sudo yum install -y git dotnet-sdk-7.0 dotnet-sdk-8.0
git clone https://github.com/JetBrains/format-ripper.git
cd format-ripper/net
dotnet restore
dotnet test -f net7.0

Hardware (IBM OpenPOWER AC922 - hypervisor level):

processor       : 127
cpu             : POWER9, altivec supported
clock           : 3800.000000MHz
revision        : 2.3 (pvr 004e 1203)

timebase        : 512000000
platform        : PowerNV
model           : 8335-GTH
machine         : PowerNV 8335-GTH
firmware        : OPAL
MMU             : Radix

.NET info:

$ dotnet --info
.NET SDK:
 Version:           8.0.100
 Commit:            57efcf1350
 Workload version:  8.0.100-manifests.2d90560f

Runtime Environment:
 OS Name:     fedora
 OS Version:  39
 OS Platform: Linux
 RID:         fedora.39-ppc64le
 Base Path:   /usr/lib64/dotnet/sdk/8.0.100/

.NET workloads installed:
 Workload version: 8.0.100-manifests.2d90560f
There are no installed workloads to display.

Host:
  Version:      8.0.0
  Architecture: ppc64le
  Commit:       5535e31a71

.NET SDKs installed:
  7.0.114 [/usr/lib64/dotnet/sdk]
  8.0.100 [/usr/lib64/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 7.0.14 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 7.0.14 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  DOTNET_ROOT       [/usr/lib64/dotnet]

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download
alhad-deshpande commented 7 months ago

@ww898 This issue has been fixed with the PR https://github.com/dotnet/runtime/pull/100337 and this fix should get released in .NET 9 preview 3 version. Hence, we can close this issue.

Below is the console output of the test case with the fix and it works fine.

(base) root@p006vm101-ub20:~/issue-on-net9# git clone https://github.com/JetBrains/format-ripper.git Cloning into 'format-ripper'... remote: Enumerating objects: 2160, done. remote: Counting objects: 100% (1017/1017), done. remote: Compressing objects: 100% (424/424), done. remote: Total 2160 (delta 624), reused 854 (delta 546), pack-reused 1143 Receiving objects: 100% (2160/2160), 65.58 MiB | 23.60 MiB/s, done. Resolving deltas: 100% (1170/1170), done. (base) root@p006vm101-ub20:~/issue-on-net9# cd format-ripper/net (base) root@p006vm101-ub20:~/issue-on-net9/format-ripper/net# dotnet restore Restore complete (11.4s)

Build succeeded in 12.6s (base) root@p006vm101-ub20:~/issue-on-net9/format-ripper/net# dotnet test -f net9.0 Restore complete (3.0s)

Build succeeded in 4.1s

Build succeeded in 0.4s (base) root@p006vm101-ub20:~/issue-on-net9/format-ripper/net#

akoeplinger commented 4 months ago

Closing the issue since it is fixed.