Open trivedigaurav opened 9 years ago
It may not be parenthesis specifically as ""("CLIA") as" indicates any-adenoma to be false." worked just fine...
On Wed, Jul 22, 2015 at 6:40 PM, Gaurav Trivedi notifications@github.com wrote:
Both wordtree and highlight span feedback fail if the span includes a parenthesis. For example
Insitu Hybridization Assays (ISH & FISH)
Error trace:
22-Jul-2015 18:36:12.279 SEVERE [http-nio-8080-exec-17] com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException The RuntimeException could not be mapped to a response, re-throwing to the HTTP container java.lang.ArrayIndexOutOfBoundsException: 89 at edu.pitt.cs.nih.backend.featureVector.Preprocess.separatePathologyHeaderFooter(Preprocess.java:302) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.getFullDSFeatureVector(ColonoscopyDS_SVMLightFormat.java:308) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.createFullDS(ColonoscopyDS_SVMLightFormat.java:96) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.createLearningFileSet(ColonoscopyDS_SVMLightFormat.java:850) at edu.pitt.cs.nih.backend.feedback.TextFileFeedbackManagerLibSVM.createLearningFiles(TextFileFeedbackManagerLibSVM.java:339) at edu.pitt.cs.nih.backend.feedback.TextFileFeedbackManager_LibSVM_WordTree.processFeedback(TextFileFeedbackManager_LibSVM_WordTree.java:78) at frontEnd.serverSide.controller.Feedback_Controller.processFeedback(Feedback_Controller.java:194) at frontEnd.serverSide.controller.Feedback_Controller.getFeedback(Feedback_Controller.java:42) at frontEnd.serverSide.WSInterface.getFeedback(WSInterface.java:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:744)
— Reply to this email directly or view it on GitHub https://github.com/trivedigaurav/emr-nlp-server/issues/12.
Sending in "ISH" alone seems to problematic:
Received span map: {fbId=0, selected=ISH, matched=ISH} ISH 22-Jul-2015 18:54:38.403 SEVERE [http-nio-8080-exec-20] com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException The RuntimeException could not be mapped to a response, re-throwing to the HTTP container java.lang.ArrayIndexOutOfBoundsException
On Wed, Jul 22, 2015 at 6:53 PM, emr-vis-nlp notifications@github.com wrote:
It may not be parenthesis specifically as ""("CLIA") as" indicates any-adenoma to be false." worked just fine...
On Wed, Jul 22, 2015 at 6:40 PM, Gaurav Trivedi notifications@github.com wrote:
Both wordtree and highlight span feedback fail if the span includes a parenthesis. For example
Insitu Hybridization Assays (ISH & FISH)
Error trace:
22-Jul-2015 18:36:12.279 SEVERE [http-nio-8080-exec-17] com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException The RuntimeException could not be mapped to a response, re-throwing to the HTTP container java.lang.ArrayIndexOutOfBoundsException: 89 at edu.pitt.cs.nih.backend.featureVector.Preprocess.separatePathologyHeaderFooter(Preprocess.java:302) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.getFullDSFeatureVector(ColonoscopyDS_SVMLightFormat.java:308) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.createFullDS(ColonoscopyDS_SVMLightFormat.java:96) at edu.pitt.cs.nih.backend.featureVector.ColonoscopyDS_SVMLightFormat.createLearningFileSet(ColonoscopyDS_SVMLightFormat.java:850) at edu.pitt.cs.nih.backend.feedback.TextFileFeedbackManagerLibSVM.createLearningFiles(TextFileFeedbackManagerLibSVM.java:339) at edu.pitt.cs.nih.backend.feedback.TextFileFeedbackManager_LibSVM_WordTree.processFeedback(TextFileFeedbackManager_LibSVM_WordTree.java:78) at frontEnd.serverSide.controller.Feedback_Controller.processFeedback(Feedback_Controller.java:194) at frontEnd.serverSide.controller.Feedback_Controller.getFeedback(Feedback_Controller.java:42) at frontEnd.serverSide.WSInterface.getFeedback(WSInterface.java:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:744)
— Reply to this email directly or view it on GitHub https://github.com/trivedigaurav/emr-nlp-server/issues/12.
— Reply to this email directly or view it on GitHub https://github.com/trivedigaurav/emr-nlp-server/issues/12#issuecomment-123896883 .
Marking this as CRITICAL as an error like this seems to corrupting the entire model, unless reset...
Heuristics for separating out headers in pathology reports are wrong. The search strings for pathology start cause an index out of bounds exceptions in some cases.
b8945764ef8126eb781aa08257367b2c7db9d0d2 fixes this for now. Need extensive testing. Also need to fix ResetDB.jar
as it still relies on the same code...
After this fix, the retraining succeeds with the warning messages like these:
Report:0022_001 does not have pseudo instances
Report:0132_000 does not have pseudo instances
Report:0190_000 does not have pseudo instances
Report:0227_000 does not have pseudo instances
Report:0384_000 does not have pseudo instances
Report:0420_000 does not have pseudo instances
Report:0489_000 does not have pseudo instances
Report:0559_000 does not have pseudo instances
Report:0576_000 does not have pseudo instances
Report:0577_000 does not have pseudo instances
Report:0608_000 does not have pseudo instances
Report:0679_000 does not have pseudo instances
Report:0694_000 does not have pseudo instances
Report:0801_000 does not have pseudo instances
Report:0808_000 does not have pseudo instances
Report:0868_000 does not have pseudo instances
Report:0910_000 does not have pseudo instances
Report:0947_000 does not have pseudo instances
Report:0984_000 does not have pseudo instances
Both wordtree and highlight span feedback fail if the span includes a parenthesis. For example
Error trace: