Generic Test Harness & Benchmarking

[rofinn]

Given the flexible workflow for building custom RBMs it would be helpful to have a generic test harness & benchmarking function for people to test their custom RBMs with.

NOTE: I can work on this.

[dfdx]

Not sure how to make a generic harness for testing RBM, ConditionalRBM and, say, future OpenCL RBM, but if you have a vision, please go ahead. With the number of possible flows and configurations we have after refactoring, tests become super important, so any generalization is heavily welcome.

[dfdx]

Do we need something behind what we already have?

[rofinn]

So, I'm probably going add some more tests to improve coverage, but there is a lot of common code between testrbm.jl and testconditional.jl. I'm thinking that if we can solidify what the public interface to an AbstractRBM should be we can put a lot of the common test code and benchmarks into a test harness function that people can use. Thoughts?

[dfdx]

Then let's consider currently exported functions as public and concentrate on them. We can add other functions later if needed.

[rofinn]

Sounds good.

[rofinn]

An initial attempt at providing generic testing & benchmarking functions for rbms can be found here

[dfdx]

Nice! Do I understand correctly that generate_dataset incorporates some patterns into data so that RBM could actually learn something?

[rofinn]

Yes. This way we can actually assess that the RBM can learn rather than just that the code runs.