Closed Mpdreamz closed 8 years ago
robertlyson
commented
9 years ago Concept looks great guys!
Happy to help. May you point a sample test? So it would be easier to follow. Also, are you going to create some kind of todo list, so we can attache PRs?
tkirill
commented
9 years ago This sounds incredible! It would be really interesting to see the result!
Mpdreamz
commented
9 years ago Hey @robertlyson and @tkirill
Currently we are working in the logical-grouping branch but we are merging that into 2.0 today, There are about 120 refactoring commits in there. I'll open another checkbox ticket tomorrow that tries to list all the (breaking) changes that are going into 2.0.
The idea behind the testing is that we now only have 1 test project (instead of the 5 we had previously) simply called Tests
If we look at an example FromAndSize.cs
You can see it uses comments that becomes part of the documentation. The idea is that we automatically take the Initializer and Fluent and use it as the documentation. The calls they go into are specified in to ClientUsage() method each UsageTest must implement.
Not only do those two drive documentation they are also both used for (de)serialization tests when running in unit test mode. In integration mode by default we do some testing on status codes and expected isvalid but we always assert on all 4 endpoints (fluent/initializer sync/async). Implementations can implement more integration tests. Methods marked with [I] are picked up when we run the tests in integration mode, and [U] when running in unit test mode.
We also allow you to write more story driven tests such as Connecting.cs which takes comments into the extreme that all make it into the documentation!. All the code in the documentation is real code that has to compile :)
At the heart of the documentation is our Litterateur where we use roslyn to visit the csharp syntax of our tests.
From the command line you can run build quicktest this will do an incremental build and runs Tests.dll in unit test mode. To run integration tests run build integrate [versionnumber] this will download and run elasticsearch versionnumber automatically. We start with a random node/cluster name and try to clean up afterwards. Obviously 2.0 is not released so right now it involves manually extracting to the right location in %APPDATA%\NEST but that should sort it self out when Elasticsearch 2.0 hits GA :+1:
They are regular xUnit 2.0 tests that can be run from your favorite visual studio test runner though (NCrunch, Resharper etc).
Both the command line tooling, documentation generation still need that last 10% polishing.
Any feedback in form of comments (or PR's :smile: ) are very much appreciated!
robertlyson
commented
9 years ago @Mpdreamz any idea what I'm missing that I can't run tests from command line?
I tried to run this command on the latest version of sources from 2.0 branch.
KodrAus
commented
9 years ago Is there a plan on how the writing of this nice big list of tests is going to be done to prevent colliding PRs? I wouldn't mind taking a look at doing some of the mapping/fields tests to get started.
robertlyson
commented
9 years ago Just in case someone will be struggling with the same issue - I've managed to fix this by reinstalling dependencies.
Mpdreamz
commented
8 years ago Closing this, will open a new issue to track documentation.
In
2.0we are redoing our test architecture so that we can we wrtite our unit/integration tests in one go while they also serve as compilable documentation thanks to some roslyn magic. We're dubbing this literate testing.The new tests (and also much of the client) are layout to mimic the official elasticsearch documentation.
This lists all the literate tests that we need to write for
2.0based of the elasticsearch documentation. We have more that deal with client concepts (inferring, failover, connectionsettings) that are outside the scope of this issue.