search

CogComp / saul

Saul : Declarative Learning-Based Programming
Other
64 stars 18 forks source link

on testing the constrained classifiers #294

Closed danyaljj closed 8 years ago

danyaljj commented 8 years ago

I'm arguing that this line in test function of the ConstrainedClassifier creates conceptual issues for testing constrained classifiers.

Consider the output of two lines from ER example: 

    println("PerConstrainedClassifier.pathToHead.get.to.getTestingInstances =  " + PerConstrainedClassifier.pathToHead.get.to.getTestingInstances.size)
    println("tokens.getTestingInstances.size =  " + tokens.getTestingInstances.size)
[info] PerConstrainedClassifier.pathToHead.get.to.getTestingInstances =  969
[info] tokens.getTestingInstances.size =  63168

this means that if you use pathToHead the number of the instances that you'd get is much smaller that the original test size (969 is suspiciously small; there might be a bug somewhere ... ). Hence testing on the result of pathToHead is not realistic. Therefore, we should never use it in the testing (hence always use clf.node.getTestingInstances).

kordjamshidi commented 8 years ago

oh this was relevant to my last issue indeed about disconnected nodes, #400