I'm writing this mostly as a reminder to myself, but also for general awareness: the Travis CI hookups for CalendarServer as currently configured do not run any unit tests. This is because I couldn't get them all to pass when initially configuring Travis (mostly for environmental reasons iirc), but instead of just running the ones that work, I run none at all. Mea culpa. I did want to do that, but the test harness doesn't have a concept of a blacklist (only a whitelist, or 'all'), so I'd need to build a list of all the tests to run, and then some automation to maintain that list as new tests are added, which seems sketchy. Probably a better solution is to add support for a skiplist to trial, or... maybe switch test harnesses. I hear tox is cool.
Travis does run CalDAVTester, which is a comprehensive over-the-wire protocol conformance and regression test suite, but that's only somewhat overlapped with the things tested by unit tests. If we had unit tests via Travis, they would have caught a problem with the upgrade to schema version 66 that just landed, because the unit tests include coverage of all the individual schema upgrades (applied in sequence), while the preparation for CalDAVTester only applies the current schema to a fresh DB; CalDAVTester itself has no visibility or opinions about the version number of that schema is.
I'm writing this mostly as a reminder to myself, but also for general awareness: the Travis CI hookups for CalendarServer as currently configured do not run any unit tests. This is because I couldn't get them all to pass when initially configuring Travis (mostly for environmental reasons iirc), but instead of just running the ones that work, I run none at all. Mea culpa. I did want to do that, but the test harness doesn't have a concept of a blacklist (only a whitelist, or 'all'), so I'd need to build a list of all the tests to run, and then some automation to maintain that list as new tests are added, which seems sketchy. Probably a better solution is to add support for a skiplist to trial, or... maybe switch test harnesses. I hear tox is cool.
Travis does run CalDAVTester, which is a comprehensive over-the-wire protocol conformance and regression test suite, but that's only somewhat overlapped with the things tested by unit tests. If we had unit tests via Travis, they would have caught a problem with the upgrade to schema version 66 that just landed, because the unit tests include coverage of all the individual schema upgrades (applied in sequence), while the preparation for CalDAVTester only applies the current schema to a fresh DB; CalDAVTester itself has no visibility or opinions about the version number of that schema is.