search

emorynlp / nlp4j-old

NLP tools developed by Emory University.
Other
60 stars 19 forks source link

Adding new values to Named Entity. #19

Open ShafeenaBasheer opened 8 years ago

ShafeenaBasheer commented 8 years ago

Hi, We need to add some more places to the existing model.What is approach.

jdchoi77 commented 8 years ago

Could you be more specific? What do you mean by "adding some more places" to which model? Thanks.

ShafeenaBasheer commented 8 years ago

Hi,

We need to add more locations(GPE) to NER model.

We tried following as the training data .but it is not working.

1 Peruvanthanam peruvanthanam NNP pos2=NN 0 root _ U-GPE.

Please help. Thanks in advance.

jdchoi77 commented 8 years ago

It's hard to tell what's not working when there is no explicit error sign I can take a look. Your training data should consist of sentences annotated with named entities (instead of individual tokens). Training a completely new model using your data should be relatively easy (please see https://github.com/emorynlp/nlp4j/blob/master/md/quickstart/train.md). Augmenting the pre-trained model is trickier but can be done by using the -p option.

ShafeenaBasheer commented 8 years ago

Hi,

We tried to augment the pre-trained model of NER by using following training files(attached ) TrainSample.zip

and run configurations -c config-train-ner.xml -t sample-trn.tsv -d sample-dev.tsv -p en-ner-old.xz -m en-ner.xz -mode ner

but we are getting following error\ 


_2016-06-15 21:08:12 INFO  BinUtils:220 - AdaGrad Mini-batch
- Max epoch: 20
- Mini-batch: 5
- Learning rate: 0.02
- LOLS: fixed = 0, decaying rate = 0.95
- RDA: 1.0E-5

2016-06-15 21:08:12 INFO  BinUtils:221 - Training: 0

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    1: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244230, NZW =  1928092, N/S =     51

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    2: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244240, NZW =  1928092, N/S =   3000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    3: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244244, NZW =  1928092, N/S =   3000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    4: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244248, NZW =  1928092, N/S =   3000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescent.expand(AdaptiveGradientDescent.java:47)
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.expand(AdaptiveGradientDescentMiniBatch.java:55)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.expand(OnlineOptimizer.java:215)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:173)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    5: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244253, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    6: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    7: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    8: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:    9: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:12 INFO  BinUtils:240 -  0:   10: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   11: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   12: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   13: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   14: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =     -1

2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   15: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   16: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =   6000

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   17: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =     -1

2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   18: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =     -1

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   19: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =     -1

java.lang.NullPointerException
    at edu.emory.mathcs.nlp.learning.optimization.AdaptiveGradientDescentMiniBatch.updateWeight(AdaptiveGradientDescentMiniBatch.java:63)
    at edu.emory.mathcs.nlp.learning.optimization.StochasticGradientDescent.trainClassification(StochasticGradientDescent.java:50)
    at edu.emory.mathcs.nlp.learning.optimization.method.AdaGradMiniBatch.trainAux(AdaGradMiniBatch.java:44)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:178)
    at edu.emory.mathcs.nlp.learning.optimization.OnlineOptimizer.train(OnlineOptimizer.java:167)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:201)
    at edu.emory.mathcs.nlp.component.template.OnlineComponent.process(OnlineComponent.java:173)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.iterate(OnlineTrainer.java:299)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:229)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:200)
    at edu.emory.mathcs.nlp.component.template.train.OnlineTrainer.train(OnlineTrainer.java:187)
    at edu.emory.mathcs.nlp.bin.NLPTrain.train(NLPTrain.java:77)
    at edu.emory.mathcs.nlp.bin.NLPTrain.main(NLPTrain.java:117)
2016-06-15 21:08:13 INFO  BinUtils:240 -  0:   20: F1:  0.00, P:  0.00, R:  0.00, L =  73, SF =  244259, NZW =  1928092, N/S =     -1_
jdchoi77 commented 8 years ago

Sorry about this; it just reminded me that the current models I released do not support this feature any more to reduce the model size. A separate module is needed to make the online learning work now :( I'll try to add this feature in the next release. Sorry again.