bethard
commented
9 years ago Comment #1 originally posted by ClearTK on 2011-02-03T18:41:07.000Z:
I like the getters/setters and am not worried about whether or not an extension is going to directly access the map or not (we could make the map private, too.) I don't like the idea that we have to initialize all of the data writers and classifiers in the initialize method. If you train on 10M instances (maybe you are doing something semi-supervised) and test on 1K, then you might find yourself loading 100 classifiers that are never used for your 1000 test instances.
One possible name is org.cleartk.classifier.multi.CleartkAnnotator (note package name). Or org.cleartk.classifier.multi.MultiCleartkAnnotator. Either way, I think it makes sense to put this code in its own package.
Original issue 226 created by ClearTK on 2011-02-03T18:31:48.000Z:
[Lee]
I was thinking since I have to do it anyway for my work, I should go ahead and write a CleartkMultiAnnotator class. The question is what can one reasonably expect from such a class? It looks like CleartkAnnotator doesn't provide much save for initialization of the classifier and data writer, and I was thinking CleartkMultiAnnotator would just provide help in dealing with collections of classifiers/ data writers. Based on conversations with Philip and my own needs, I was thinking that the pieces that differentiate CleartkMultiAnnotator from CleartkAnnotator would look something like this:
public abstract class CleartkAnnotator<KEY_TYPE, OUTCOME_TYPE> extends JCasAnnotator_ImplBase implements Initializable {
}
Can you think of anything else that would be useful? I know when Philip implemented something similar for his own purposes he had an output directory configuration parameter so that he could squirrel away the classifiers and data writers in jar files.
[Steve] Thanks, this sounds like a great contribution!
This looks like basically what I had in mind, except I would just drop the generics for KEY_TYPE and use String everywhere. Then you can have a String[] PARAM_CLASSIFIER_NAMES @ConfigurationParameter, and the DataWriters can all delegate to PARAM_OUTPUT_DIRECTORY/name. Within those subdirectories, you should be able to package and unpackage jars using the usual ClearTK machinery.
By the way, I think you probably don't need the getter methods if the Maps are protected. No one but subclasses should really be asking for the DataWriters and Classifiers here, right?
[Philip] The getters can be protected. In my solution, I used getClassifier(keyType) to get a classifier if it was already available. If not, then it would initialize the classifier. I like doing this because you may not need every classifier that's been created for/from the training data when you are running it on your evaluation data because you may (perhaps inadvertently) have some key types that are very sparse and never appear in your evaluation data.
I agree that KEY_TYPE may be unnecessary and String may suffice. Although, I can see that a boolean would be quite reasonable for some scenarios.
I would rather not use the word "key" or "type" for the naming the key of the map as they are both overloaded terms. Maybe "name"?
Finally, it would be nice if we were able to abstract out the output directory as a parameter to this analysis engine as we do with CleartkAnnotator. I don't want to wire in our jar-based solution into this new class.
[Lee] I'm leaning toward just using String like Steve suggested. If someone really needed the boolean condition they could just create appropriate strings to describe their classifiers.
As per the getters, I go back and forth on whether or not they are needed. They are handy for the on-the-fly creation like Philip described, but they also muddle the semantics when using these classifier/datawriter maps. It's not clear when writing an extension of this proposed class whether one should use the maps directly or should only access them through the getters.
Lastly, does anyone have a good name for this class?