Improve test case code coverage

[phillipod]

So the existing code coverage from testing (https://github.com/quanah/net-ldapapi/blob/master/net-ldapapi/trunk/test.pl) is:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                      6.4    2.6    n/a    n/a    n/a    n/a    5.0
blib/lib/Net/LDAPapi.pm        18.4    9.8   19.2   21.9   52.0  100.0   18.1
Total                          14.6    7.5   19.2   21.9   52.0  100.0   14.4
---------------------------- ------ ------ ------ ------ ------ ------ ------

or 14.4% code coverage.

So I've started adding BDD testing.

To see what I'm doing: https://github.com/quanah/net-ldapapi/compare/master...phillipod:test-coverage

For what I've done so far, I've got:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                      6.4    2.6    n/a    n/a    n/a    n/a    5.0
blib/lib/Net/LDAPapi.pm        12.9    6.9   19.2   17.1   52.0  100.0   13.9
Total                          10.8    5.5   19.2   17.1   52.0  100.0   11.4
---------------------------- ------ ------ ------ ------ ------ ------ ------

Currently only 11.4% - but before I push on too far, any comments?

I like the BDD style - the test cases are easily understandable, etc etc.

But perhaps Test::BDD::Cucumber is an odd requirement - perhaps only run the BDD tests in a developer mode, keeping test.pl for installation verification?

Or I'm happy to consider an alternative approach..

[phillipod]

As an update the following scenarios currently have test cases in my tree:

• Feature: Binding to the directory
  • Scenario: Can bind anonymously
  • Scenario: Can bind with simple authentication
  • Scenario: Can asynchronously bind anonymously
  • Scenario: Can asynchronously bind with simple authentication
• Feature: Searching the directory
  • Scenario: Can search
• Feature: Querying the directory for my identity
  • Scenario: Can query identity with anonymous authentication
  • Scenario: Can query identity with simple authentication

With these scenarios, and only these scenarios, code coverage is:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     11.1    6.2    n/a    n/a    n/a    n/a    9.3
blib/lib/Net/LDAPapi.pm        20.6   11.9   26.9   22.6   51.5  100.0   20.1
Total                          17.5   10.1   26.9   22.6   51.5  100.0   17.0
---------------------------- ------ ------ ------ ------ ------ ------ ------

[phillipod]

With the addition of asynchronous search testing, it's now at 21.5% code coverage, mostly due to a jump in code paths exercised in the XS component.:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     20.1   16.5    n/a    n/a    n/a    n/a   18.8
blib/lib/Net/LDAPapi.pm        23.3   14.6   26.9   25.9   50.5  100.0   22.6
Total                          22.3   15.2   26.9   25.9   50.5  100.0   21.5
---------------------------- ------ ------ ------ ------ ------ ------ ------

[quanah]

As long as the modules aren't required for building/installing Net::LDAPapi (i.e., it's optional), I'm fine with it. Not everyone wants to run a test suite. ;)

[phillipod]

Sounds fine to me.

Next up will be write tests (add/modify/modrdn/delete) and possibly some options tests.

Current status:

• Feature: Binding to the directory
  • Scenario: Can bind anonymously
  • Scenario: Can bind with simple authentication
  • Scenario: Can asynchronously bind anonymously
  • Scenario: Can asynchronously bind with simple authentication
• Feature: Executing extended operations against the directory
  • Scenario: Can match identities retrieved with native whoami and using extended operations with anonymous authentication
  • Scenario: Can match identities retrieved with native whoami and using extended operations with simple authentication
  • Scenario: Can asynchronously match identities retrieved with native whoami and using extended operations with anonymous authentication
  • Scenario: Can asynchronously match identities retrieved with native whoami and using extended operations with simple authentication
• Feature: Searching the directory
  • Scenario: Can find objects that exist within the directory
  • Scenario: Can asynchronously find objects that exist within the directory
  • Scenario: Can find objects that exist within the directory and iterate them with next_entry and next_attribute
  • Scenario: Can find objects that exist within the directory and iterate them with next_entry and entry_attribute
  • Scenario: Can find objects that exist within the directory and iterate them with result_entry and next_attribute
  • Scenario: Can find objects that exist within the directory and iterate them with result_entry and entry_attribute
• Feature: Querying the directory for my identity
  • Scenario: Can query identity with anonymous authentication
  • Scenario: Can query identity with simple authentication
  • Scenario: Can asynchronously query identity with anonymous authentication
  • Scenario: Can asynchronously query identity with simple authentication

Coverage:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     22.4   14.7    n/a    n/a    n/a    n/a   19.6
blib/lib/Net/LDAPapi.pm        36.0   22.1   34.6   41.7   47.2  100.0   32.9
Total                          31.6   19.8   34.6   41.7   47.2  100.0   29.1
---------------------------- ------ ------ ------ ------ ------ ------ ------

[quanah]

At least with OpenLDAP, SASL/EXTERNAL is always available, if you want to add some SASL tests into the mix.

[phillipod]

Good idea. I use GSSAPI myself, so I may also throw in an optional test for that.

As an update:

Add/Modify/Delete/Rename tests are complete.

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     45.4   36.1    n/a    n/a    n/a    n/a   42.0
blib/lib/Net/LDAPapi.pm        48.5   33.6   34.6   49.5   47.2  100.0   43.7
Total                          47.5   34.4   34.6   49.5   47.2  100.0   43.2
---------------------------- ------ ------ ------ ------ ------ ------ ------

[phillipod]

Getting a test working for the syncrepl consumer would likely be the next biggest win in terms of code coverage. Plus it'll provide verification for work in #16

[phillipod]

I think for a 3.0.4 milestone, the goals I have for more test coverage are:

• syncrepl consumer
• controls
• compare
• get options / set options
• sasl binding

And of course, any bugfixes identified in creating the above.

At that point, any work on 3.1 should follow a test-case first model - if a bug is found or enhancement required in a feature without a test case, a test case would be written first.

[phillipod]

There are now tests for both the syncrepl consumer as well as server side controls in general

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     56.5   42.8    n/a    n/a    n/a    n/a   51.5
blib/lib/Net/LDAPapi.pm        64.9   43.5   38.4   59.6   45.5  100.0   56.1
Total                          62.2   43.3   38.4   59.6   45.5  100.0   54.8
---------------------------- ------ ------ ------ ------ ------ ------ ------

[phillipod]

And now tests for compare

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     58.1   42.8    n/a    n/a    n/a    n/a   52.5
blib/lib/Net/LDAPapi.pm        68.6   47.0   42.3   61.6   45.1  100.0   59.3
Total                          65.2   45.7   42.3   61.6   45.1  100.0   57.4
---------------------------- ------ ------ ------ ------ ------ ------ ------

.

[phillipod]

With get/set option tests as well as a new test for get_all_entries():

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     68.6   53.3    n/a    n/a    n/a    n/a   63.0
blib/lib/Net/LDAPapi.pm        70.2   47.2   42.3   64.4   44.7  100.0   60.4
Total                          69.7   49.3   42.3   64.4   44.7  100.0   61.2
---------------------------- ------ ------ ------ ------ ------ ------ ------

[phillipod]

I've added SASL EXTERNAL testing. I'll leave GSSAPI testing for 3.1.

Note that my SASL EXTERNAL test suite relies on UID/GID mapping via the UNIX socket handling the ldapi protocol. I've added bind and whoami tests (synchronously as well as asynchronously). Currently due to #25, these two asynchronous tests are failing - the synchronous tests are working.

I've got to do a bit of cleanup for the test suite integration, but I consider that the scope for 3.0.4 for the test suite has been met.

22 and #25 are both in scope for 3.0.4 and once I've fixed them, as well as the test suite integration, I'll submit a pull request for the test suite. Then I think it'll be down to the release tasks before 3.0.4 can start to make its way out the door.

I imagine you've got some testing you'll want to do as well, etc.

[phillipod]

Okay. So #22 is closed off, and I think #25 should bump to 3.1.x. I've changed it so the test suite is optional (and disabled by default).

So the question is - how do you want it? Multiple pull requests, like one per feature? A single commit with everything? Or the whole git history as it is?

[phillipod]

For interests sake, here's the current complete list of features and scenarios:

• Feature: Adding entries to the directory
  • Scenario: Can add a new entry to the directory
  • Scenario: Can asynchronously add a new entry to the directory
• Feature: Binding to the directory
  • Scenario: Can bind anonymously
  • Scenario: Can bind with simple authentication
  • Scenario: Can bind with sasl authentication
  • Scenario: Can asynchronously bind anonymously
  • Scenario: Can asynchronously bind with simple authentication
• Feature: Comparing values to values of attributes of entries within the directory
  • Scenario: Can compare an attribute on an entry within the directory
  • Scenario: Can asynchronously compare an attribute on an entry within the directory
• Feature: Deleting entries from the directory
  • Scenario: Can remove an entry from the directory
  • Scenario: Can asynchronously remove an entry from the directory
• Feature: Executing extended operations against the directory
  • Scenario: Can match identities retrieved with native whoami and using extended operations with anonymous authentication
  • Scenario: Can match identities retrieved with native whoami and using extended operations with simple authentication
  • Scenario: Can asynchronously match identities retrieved with native whoami and using extended operations with anonymous authentication
  • Scenario: Can asynchronously match identities retrieved with native whoami and using extended operations with simple authentication
• Feature: Updating attributes of entries within the directory
  • Scenario: Can add a new attribute to an entry within the directory
  • Scenario: Can modify an attribute on an entry within the directory
  • Scenario: Can remove an attribute on an entry within the directory
  • Scenario: Can asynchronously add a new attribute to an entry within the directory
  • Scenario: Can asynchronously modify an attribute on an entry within the directory
  • Scenario: Can asynchronously remove an attribute on an entry within the directory
• Feature: Control of options used to configure the LDAP library
  • Scenario: Can set and read back options
• Feature: Renaming entries within the directory
  • Scenario: Can rename an entry within the directory
  • Scenario: Can asynchronously rename an entry within the directory
• Feature: Searching the directory
  • Scenario: Can find objects that exist within the directory
  • Scenario: Can asynchronously find objects that exist within the directory
  • Scenario: Can find objects that exist within the directory with a timeout
  • Scenario: Can asynchronously find objects that exist within the directory with a timeout
  • Scenario: Can find objects that exist with the directory and iterate them with next_entry and next_attribute
  • Scenario: Can find objects that exist with the directory and iterate them with next_entry and entry_attribute
  • Scenario: Can find objects that exist with the directory and iterate them with result_entry and next_attribute
  • Scenario: Can find objects that exist with the directory and iterate them with result_entry and entry_attribute
• Feature: Using server controls to control results
  • Scenario: Can use the Server Side Sort control
• Feature: Listening for changes within the directory with syncrepl
  • Scenario: Can listen for changes within the directory
• Feature: Querying the directory for my identity
  • Scenario: Can query identity with anonymous authentication
  • Scenario: Can query identity with simple authentication
  • Scenario: Can query identity with sasl authentication
  • Scenario: Can asynchronously query identity with anonymous authentication
  • Scenario: Can asynchronously query identity with simple authentication

That's 12 features, with 39 scenarios, for a total of 339 steps.

And here's the code coverage status:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
LDAPapi.xs                     70.5   52.9    n/a    n/a    n/a    n/a   64.1
blib/lib/Net/LDAPapi.pm        73.8   48.9   46.1   72.1    4.3  100.0   64.1
Total                          72.7   50.2   46.1   72.1    4.3  100.0   64.1
---------------------------- ------ ------ ------ ------ ------ ------ ------

[quanah]

I think a single commit with everything is fine.

[phillipod]

Any further major development on this is likely to target a 3.1.x release. Bumping milestone.