consider running all libraries unit tests with bogus machine culture

[danmoseley]

https://github.com/dotnet/runtime/pull/92924#issuecomment-1744677940

This was a case where the product was not using invariant culture to parse a date.

It's probably worth having a look to see whether any unit tests throw up similar issues. We do run at least inner loop on es-ES in CI I believe. But I'm not sure this includes outerloop, and es-ES is sufficiently similar to en-US that it does not catch certain kinds of issues (eg., historically we've had several tests failed in Russian machine culture, and they were community reported)

suggestion: run all the unit tests including outer loop in a different culture, ideally with a different calendar, etc. or even a custom culture that is different to neutral culture in every respect -- calendar, date format, numeric specifier, etc. Every test should pass.

I've done this historically from time to time but I'll put this up for grabs this time

cc @filipnavara

[ghost]

Tagging subscribers to this area: @dotnet/area-meta See info in area-owners.md if you want to be subscribed.

Issue Details

---

https://github.com/dotnet/runtime/pull/92924#issuecomment-1744677940 This was a case where the product was not using invariant culture to parse a date. It's probably worth having a look to see whether any unit tests throw up similar issues. We do run at least inner loop on es-ES in CI I believe. But I'm not sure this includes outerloop, and es-ES is sufficiently similar to en-US that it does not catch certain kinds of issues (eg., historically we've had several tests failed in Russian machine culture, and they were community reported) suggestion: run all the unit tests including outer loop in a different culture, ideally with a different calendar, etc. or even a custom culture that is different to neutral culture in every respect -- calendar, date format, numeric specifier, etc. I've done this historically from time to time but I'll put this up for grabs this time cc @filipnavara

Author:	danmoseley
Assignees:	-
Labels:	`area-Meta`, `help wanted`
Milestone:	-

[danmoseley]

@filipnavara sounded like you might do this at some point? or up for grabs?

[filipnavara]

It's not exactly on the top of my list but I do have a machine with such a weird setup so I will likely eventually get to do it. Feel free to assign me.

[danmoseley]

Cool - done. I wasn't expecting, just sounded like you planned to 😀

[MichalPetryka]

ideally with a different calendar

I'd say that cultures with a comma or the arabic ٫ would be the most important. We've had one 3rd party library fail with double parsing but ONLY on Czech culture once.

I think that having a weekly pipeline for running with a few weird cultures would be the solution here (like Czech, Arabic, Thai, Japanese).

[filipnavara]

I run all the inner loop tests on macOS Sonoma with the Saudi Arabian culture with non-Gregorian calendar. There were no additional culture-related failures. There was one spurious failure in System.Net.Http.Functional.Tests.HttpMetricsTest_Http11_Async.RequestDuration_ConnectionClosedWhileReceivingHeaders_Recorded, and the OpenSSL and ODBC tests were skipped due to lack of native libraries.

[ericstj]

@dotnet/area-system-globalization @dotnet/area-infrastructure-libraries to weigh in on this.

[tarekgh]

Running the libraries tests with random culture is a good idea. We can think of having a CI leg which can force a random culture on the machine before running.

[danmoseley]

As a simple step we could consider replacing our regular en-ES run (I assume we still do) with one using a culture that has more differences to en-US. OS culture also changes things like OS messages.