search

alex-pirozhenko / sklearn-pmml

A library that allows serialization of SciKit-Learn estimators into PMML
MIT License
70 stars 17 forks source link

Testing framework #16

Open jnothman opened 8 years ago

jnothman commented 8 years ago

This project currently does not have enough tests to ensure the faithfulness of the exported model to the original. A test harness should be written that compares the output of the original predictor against its exported model run through an external PMML execution engine.

NeverNude commented 8 years ago

See jpmml_test, and examples of using it sklearn-pmml/sklearn_pmml/convert/test/test_decisionTreeClassifierConverter.py. It runs the original sklearn model against the exported model via JPMML (invoking via mvn) and compares the output.

jnothman commented 8 years ago

Oh, very good! Sorry I didn't see it.

ngould commented 8 years ago

Hey all! I just started familiarizing myself with this project today. I'm running on OSX, with Java 1.7.0_78, and Maven 3.3.3. Running py.test results in two failures and one error.

Here's a gist of my py.test output. https://gist.github.com/ngould/09a236af202e0ab15653

Wondering if you're seeing the same thing, or if I have a problem with my setup. I'm not all that familiar with Java and Maven, so this could be due to ignorance on my part. Any ideas?

Also, are log outputs like this anything to be concerned about?

2015-08-29 19:30:09,884 Unable to convert DOM node output at [UNAVAILABLE] to binding
2015-08-29 19:30:09,885 Unable to convert DOM node x1 at [UNAVAILABLE] to binding
2015-08-29 19:30:09,885 Unable to convert DOM node x2 at [UNAVAILABLE] to binding

I'd love to help contribute to the project. Is there a mailing list, IRC channel, etc?

NeverNude commented 8 years ago

Hey @ngould! There isn't currently a mailing list, IRC channel etc... we've just been using github issues/ to communicate, though as it grows we should make something more usable. Looking at that it seems as if I set the target to be 1.8 for the JPMML integration tests, rather than 1.7 which you are running on. I'll change the target to be 1.7, since that's all that's required.