skip tests run as "non-packaged" (non-rpm) validation

[Vishwanatha-HD]

Added changes to skip the dotnet regular tests which are not required to run during cross build SDK validation.

[Vishwanatha-HD]

@omajid. please review my changes and merge it, if the changes are fine.

[tmds]

// skip during cross-bld-sdk/tarball validation

@Vishwanatha-HD this doesn't describe why the test fails. I've added some questions as I like to understand better how the test fail in the "non-packaged" environment?

[Vishwanatha-HD]

Hi @tmds.. Omair and me had a discussion about the failures and please find his explanation below on why the tests fail when we try to validate the cross-build SDKs.. We mean "non-packaged" when we try to validate just the tarball and not the complete RPMs..

The errors fall into a few buckets:

Failing tests >>>> -> publish-aspnet-selfcontained -> publish-dotnet-selfcontained -> restore-with-rid -> openssl-alpn -> system-openssl:

• The above tests fail because they expect to be testing a source-built SDK and a cross-built SDK is different enough
  • For example, doesn't know about rhel.9-s390x RID
  • The cross-built SDK is portable, and doesn't link to some system libraries that the tests want to poke through

Failing tests >>>> -> bash-completion -> man-pages -> tools-in-path

• The above tests fail because they expect to be testing a packaged SDK, which has features like bash-completion enabled, which isn't a part of the .NET SDK tarball.

Failing tests >>>> -> file-permissions -> rsa-pkcs-openssl -> sha1-validation

• The above tests fail because they are testing Red Hat-specific changes, such as whether https://github.com/omajid/dotnet-runtime/commit/076687f5f9e7e1fce24f33f498b4e03c4150108e has been applied to the SDK.

Please let me know if you need more info and we will able to provide them.. Thanks..

[tmds]

why the tests fail when we try to validate the cross-build SDKs..

fyi, in hydra CI we don't run the regular tests against the cross-built SDKs. The cross-built SDK jobs provide SDKs for native builds. And those builds run the tests.

For example, doesn't know about rhel.9-s390x RID

The cross-built SDK for .NET 9 gets built with a rid of linux-390x and provides assets under that rid. Rather than skipping these tests, they can run if they are using the appropriate SDK rid.

The cross-built SDK is portable, and doesn't link to some system libraries that the tests want to poke through

Can you be explicit about this in the comments? I don't think non-packaged conveys these tests are skipped because the build is portable. Perhaps you could use something else, like skipWhen: portable.

Failing tests >>>>

For these other tests, which should be ones that already have vmr-ci but where we are now adding now for the same reason: non-packaged.

It would be nice if we can avoid this duplication by either using vmr-ci or non-packaged, but not add them both.

For the managed-symbols-available test you can either also extract the symbols tarball so the test would pass, or add a new dedicated trait like: skipWhen: no-symbols.

Note that the test runner can be given multiple traits to skip, so you can set it to skip test matching any of: vmr-ci/non-packaged, portable, and no-symbols.

[Vishwanatha-HD]

@tmds and @omajid, I have addressed all the code review comments and made the corresponding changes. Please take a look and merge the CL if its fine.. thanks

[omajid]

This is great! Thank for you this really nice cleanup!

[Vishwanatha-HD]

@omajid, I took a look at your changes to run the testcase with SDK runtime-ids rather than the RPM ones.. The changes are looking good.. Thanks for doing this for me.. !!