search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
13.86k stars 3.37k forks source link

[CI][C#] Verification jobs for C# are currently failing #43058

Closed raulcd closed 2 days ago

raulcd commented 3 days ago

Describe the bug, including details regarding any error messages, version, and platform.

Tests are failing with several errors that seem to be related to (see the logs for more details):

 ----- Inner Stack Trace -----
   at Apache.Arrow.Tests.CDataSchemaPythonTest.PythonNet..ctor() in /Users/runner/work/crossbow/crossbow/arrow/csharp/test/Apache.Arrow.Tests/CDataInterfacePythonTests.cs:line 48
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
  Failed Apache.Arrow.Tests.CDataSchemaPythonTest.ImportSchema [1 ms]
  Error Message:
   Class fixture type 'Apache.Arrow.Tests.CDataSchemaPythonTest+PythonNet' threw in its constructor

Job failures:

The failures started when those commits were merged: https://github.com/apache/arrow/compare/ff9921ffa89585be69ae85674bb365d03cb22ba4...f904928054fad89360d83015db5c23ac1ef86d05 There are several bump versions from dependabot on those that involves C#.

Component(s)

C#, Continuous Integration

raulcd commented 3 days ago

@CurtHagenlocher @adamreeve @eerhardt any idea what is happening? this is a blocker for 17.0.0

CurtHagenlocher commented 3 days ago

The inner exception seems to be "Xunit.SkipException : PYTHONNET_PYDLL not set; skipping C Data Interface tests". This would happen if the environment variable PYTHONNET_PYDLL was not set on the VM image and is independent of any changes to product code. Something must have changed in the VM configurations?

adamreeve commented 3 days ago

It looks like these tests are supposed to be skipped in the verification jobs: https://github.com/apache/arrow/blob/6680dcfbef4234782484cd8c2c3e0ce0cf9e9914/csharp/test/Apache.Arrow.Tests/CDataInterfacePythonTests.cs#L39 https://github.com/apache/arrow/blob/6680dcfbef4234782484cd8c2c3e0ce0cf9e9914/csharp/test/Apache.Arrow.Tests/CDataInterfacePythonTests.cs#L42-L44

Maybe the TEST_CSHARP environment variable is no longer being set, or it's now set to something other than "1"?

adamreeve commented 3 days ago

Actually the logs show that these are throwing an Xunit.SkipException rather than an Exception so it seems like the check for inVerificationJob is correct, but the skip is being incorrectly treated as an error.

So I think this is probably due to the either #41842 or #41848

adamreeve commented 3 days ago

I can reproduce this locally and the tests pass (are skipped properly) if I revert #41842

adamreeve commented 3 days ago

I've created a bug for xunit: https://github.com/xunit/xunit/issues/2965

CurtHagenlocher commented 3 days ago

I guess I don't have a firm grasp on the semantics of TEST_CSHARP or GITHUB_ACTIONS. These tests should certainly be run for at least some of the CI/checkin tests but perhaps not for all of them?

In any event, the xunit change is clearly a bug.

raulcd commented 2 days ago

I can reproduce this locally and the tests pass (are skipped properly) if I revert #41842

Should we revert this temporarily for the release and create a follow up issue to update once the downstream issue is fixed?

adamreeve commented 2 days ago

Should we revert this temporarily for the release and create a follow up issue to update once the downstream issue is fixed?

Yes I think that makes sense, I've made #43074 to revert the update and #43076 for the follow up.

kou commented 2 days ago

Issue resolved by pull request 43074 https://github.com/apache/arrow/pull/43074