PROPOSAL: UI refresh

[kentcb]

Hi,

Before I invest too much time in this, I wanted to open this up for discussion via this PR.

What I'm proposing is a fairly radical UI refresh with one tenet at its core: to reduce friction. That is, make it as quick and simple as possible to work in a test-driven fashion when running tests on the simulator or device.

This PR contains a very early proof-of-concept that looks like this:

These are the specific things I want to do to make the test runner more useful:

• get straight to the tests. The above screen would be loaded straight away - no intermediate view with assembly names or whatever
• powerful filtering. For example, being able to type any of these (and combinations thereof):
  • foo : finds all tests with "foo" anywhere in their fully-qualified name
  • assembly:MyAssembly : finds only those tests in the MyAssembly assembly
  • class:MyClass : finds only those tests in the MyClass class
  • duration:>100 : finds only those tests with a duration greater than 100ms
  • status:passed : finds only those tests that passed
  • status:notrun,failed : finds only those tests that are yet to run or have failed
• saving state for next run. This would allow you to set the filter once for a particular piece of functionality you're working on, and each time you re-start the simulator (or device) it would pick up and apply that filter, saving you time. Same would apply for other options
• ordering options. Order by test name, class name, assembly name, duration in ascending/descending order. Where relevant, could maybe include grouping (e.g. when ordering by class name, tests could be grouped by their class)
• other options. It seems as though options like execute-on-startup are not currently exposed. It would be great to expose them too

So, I'd love to hear people's thoughts on this. Having played about with the code for a day or so I can see that it won't be as simple as I'd hoped (mainly because we can't use a reactive approach due to restrictions on 3rd party libs) but I think it's probably all doable (haven't looked into grouping yet).

It's also not clear to me whether the code base needs to remain platform-agnostic. It seems as though some code attempts to be compatible with other platforms (e.g. android) but other code does not.

[clairernovotny]

Looping in @aarnott @dsplaisted and @bradwilson to see if they have additional thoughts.

I'll say this - the current UI is largely based on a mash of the Xamarin NUnit runners for iOS and Android. I have no strict love of it, mostly that it was a starting point in adapting the xUnit discovery/execution and making it available for iOS/Android.

I fully agree that the UI can use a redesign and I like all that you're proposing so far.

My main hesitation when it comes to 3rd party libraries is really just around "scariness"/friction. When someone goes to create a bootstrap/runner app, would it be imposing too much to also pull in the Rx references? My gut says yes, but I'd be curious to hear what others think.

[bradwilson]

We abandoned the old GUI runner from v1.x because we didn't have time for it. From time to time, people ask about GUI runners for v2.x. If this could be adapted to a desktop runner, too, I think the community would be pretty happy about that. :)

[clairernovotny]

@bradwilson :+1: Even if a desktop runner can't use Xamarin Forms, I don't think it'd be hard to extend the view models and discovery/execution code to run there.

[AArnott]

It looks like a great start. And :+1: to implementing with view models so that it can be easily adapted to Xamarin.Forms, Windows Xaml, WPF, etc.

[kentcb]

OK, so assuming we were to increase the scope and attempt to create a common base implementation for GUI runners, what are the thoughts on third party libraries? I think I remember reading somewhere (can't find it now) that dependencies on third party libraries are forbidden...?

If so, that's fine but I'm nonetheless interested in the reasoning for that. Is it because others would find it more difficult to contribute, or because of assembly resolution problems when executing test code that also has dependencies on those libraries?

[clairernovotny]

I don't have any issues with using 3rd party libraries for self-contained apps like a desktop runner. My only concern with having too many external libs was for the device runner, where adding the xunit runner would also then pull in tons of dependencies. I thought that might be a bit off-putting and scare people away. That said, I could be overthinking it and people may not care. I honestly don't know what the best approach is for having external libs in the device runner app.

I was thinking that to keep the footprint pretty simple, we could split the current shared code project into two: one for the runner & view models and the other for the Xamarin Forms views/pages. Both shared projects would be pulled into the current XUnit.Runner.* projects so that it outputs one combined assembly per platform.

For WPF/Desktop we'd just pull in the view model shared code project and create one or more other shared/library projects that have the specific views in them.

Thoughts?

[clairernovotny]

Closing this out for now as the code has changed substantially. I still am very interested in any UI refreshes.

The current ViewModels have no Xamarin forms dependency and as can be seen in the UWP version, a XAML-based UI can be created within a few hours.