search

SwiftAndroid / swift

Port of Apple's reference Swift toolchain to Android; doesn't quite work yet
Apache License 2.0
720 stars 32 forks source link

Get tests running #6

Closed zhuowei closed 8 years ago

zhuowei commented 8 years ago

test/lit.cfg needs to have a way to push executables to Android and get stdout, stderr back correctly. Also, linker flags in lit.cfg needs to be tweaked to work on Android.

LLVM lit had a patch for running tests on an Android device but the patch was never merged. :(

The current plan is to make a small wrapper script that pushes to the device and use it in target_run

zhuowei commented 8 years ago

Some tests can run now, but the standard test framework spawns a subprocess to run tests in parallel, so need to implement subprocess.

modocache commented 8 years ago

Some tests can run now

Can you explain how you're running tests? I'd like to work on getting tests to pass on this fork.

modocache commented 8 years ago

Oh, I see 0079df115500e834d7c6fb40276a2f010ffee2d7. I'll try using that--thanks!

modocache commented 8 years ago

Here are my results for swift-linux-x86_64/test-android-armv7: https://gist.github.com/modocache/cf888cf869f06207c262

tl;dr:

  Expected Passes    : 1455
  Expected Failures  : 67
  Unsupported Tests  : 648
  Unexpected Passes  : 1
  Unexpected Failures: 23
modocache commented 8 years ago

Here are my results for swift-linux-x86_64/test-linux-x86_64: 

Testing Time: 70.10s
  Expected Passes    : 1535
  Expected Failures  : 76
  Unsupported Tests  : 583

1 warning(s) in tests.

I'll look into (a) fixing the failures on swift-linux-x86_64/test-android-armv7, and (b) reducing the number of unsupported tests for that test suite.

modocache commented 8 years ago

Tests are running pretty well. There are currently less than 10 failures across the test and validation-test suites.

To make it easier to run tests, I moved the test runner to the main Swift repository and added documentation on how to use it: https://github.com/SwiftAndroid/swift/commit/5b4e565b5be35147047072c49891ee287ef06d3e

We can track test failures in separate issues, but the overall problem of running tests has been addressed, so I'll close this issue.

zhuowei commented 8 years ago

That's awesome! thank you so much.

modocache commented 8 years ago

No, you're awesome! :+1: Thanks for the initial work in https://github.com/SwiftAndroid/swift/commit/0079df115500e834d7c6fb40276a2f010ffee2d7, it got me 99% of the way there.