Validation tool should work at the media level.

[cjcodeproj]

The validation tool should work at the media level, for the purpose of checking all data.

[cjcodeproj]

The validation code should be an extensible framework for running tests against the data.

Any class structure under media.data should be testable by a separate validation class. Those classes will probably sit under media.validation.

Ex, validating data in the class media.data.media.contents.generic.catalog.Catalog() would be validated against a class called media.validator.media.contents.generic.catalog.Catalog().

Rules:

• Validator classes will probably all be derived from a root abstract class.
• There will be classes that validate data, and classes that run the validation process.
• The classes that run the validation process should be able to select a different validation backend, if desired.
• Validation should be possible from the Python shell, as well as a command line tool.
• There should be common classes for identifying scores, and faults.
• Validation framework should understand some data is optional, and a lack of data is not a fault; but there should be an option to flag missing optional data that would benefit a record.
• The scoring should be dynamic, where a perfect case is a 10.0. That means the code has to dynamically adjust the score based on the extent of the data, and what validation classes are out there. IE, if there are ten validation tests, then each fault would decrease the perfect score of 10.0 by 1.0.
• Faults should contain a severity level.

[cjcodeproj]

Design Notes:

The validator class will probably operate as follows:

1. Initialization of object.
2. Loading a class which references a suite of testing classes to also be loaded.
3. Loading of media objects.
4. Run suite of applicable tests against every media object. IE, only run the tests suitable to the loaded media/
5. Return an object that contains all scoring information.

[cjcodeproj]

Additional features:

• The ability to run validation against a "level", so you can focus on important tests, skip tests that are less important, or take longer.
• Keep a tally count of tests that are run vs. skipped, and failed vs. passed.

[cjcodeproj]

Additional features/notes:

• The scoring mechanism is still a work in progress
• For each piece of media, there should be a way to track each individual test that is run, regardless of whether it passes or fails.

[cjcodeproj]

Definitions:

• A TestResult is the outcome of a single test. It contains the name of the test, the max possible score, the actual score, the result code, and any fault reports (if any).
• A Tray is a container object that contains the media object, and a Status object which is where all of the TestResult objects are collected.
• The Status object is what will generate overall reports, and report on the overall passing or failure of the media object.

[cjcodeproj]

Commit: https://github.com/cjcodeproj/medialibrary/commit/ec9315bb21db136e50f08ec7da3b1fbb51e030a0

[cjcodeproj]

Framework has been built:

Main validation tool is now media.tools.media.validate

Notes:

• Lots of small classes, which lead to a few plint errors of R0903.
• The commit was mostly made to get the code out there. It works, but I feel like part of the framework is still kind of beta-ish.
• More tests can always be added, and the framework could be modified down the road to simplify it.

Operation of the validator is pretty easy, from a high-level code perspective.

• The Validator object is launched and initialized.
• A suite of tests can be loaded (customizable, but with a default)
• Media objects are loaded
• Tests are run, results are returned.

[cjcodeproj]

Pull request (merged).

https://github.com/cjcodeproj/medialibrary/pull/129