When running s-pipes on windows query comments cannot be extracted for s-pipes scripts with linux line endings, see cz.cvut.spipes.util.QueryUtils.getQueryComment:
public static String getQueryComment(String query) {
String comment = query.split(System.lineSeparator())[0];
if (comment.matches("\\s*#.*")) {
return comment.split("\\s*#\\s*", 2)[1];
}
return null;
}
The query string is split by System.lineSeparator() which on windows is equal to \r\n (CRLF) but the query string contains linux line endings, i.e. \n (LF).
Possible solution:
A possible solution would be to replace System.lineSeparator() with "[\r\n]". This should work properly for windows, linux and mac line endings.
Similar issues
There is one more place where System.lineSeparator() is used and it may cause similar issues:
cz.cvut.spipes.modules.TabularModule.executeSelf
CsvPreference csvPreference = new CsvPreference.Builder(
quoteCharacter,
delimiter,
System.lineSeparator()).build();
Issue cause
Issue found when debugging exception thrown by s-pipes. The exception was thrown when running s-pipes on windows with ava-model modules checked out with linux line endings.
org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [dispatcher] in context with path [/s-pipes] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause
java.lang.NullPointerException
at java.base/java.util.Comparator.lambda$comparing$77a9974f$1(Comparator.java:469)
at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
at java.base/java.util.TimSort.sort(TimSort.java:220)
at java.base/java.util.Arrays.sort(Arrays.java:1515)
at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:353)
at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.modules.ApplyConstructModule.executeSelf(ApplyConstructModule.java:116)
at cz.cvut.spipes.modules.AbstractModule.execute(AbstractModule.java:73)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:92)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.lambda$_executePipeline$6(ExecutionEngineImpl.java:80)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1239)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at cz.cvut.spipes.engine.ExecutionEngineImpl._executePipeline(ExecutionEngineImpl.java:80)
at cz.cvut.spipes.engine.ExecutionEngineImpl.executePipeline(ExecutionEngineImpl.java:30)
at cz.cvut.spipes.rest.SPipesServiceController.runService(SPipesServiceController.java:204)
at cz.cvut.spipes.rest.SPipesServiceController.processServicePostRequest(SPipesServiceController.java:156)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:652)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Issue description
When running s-pipes on windows query comments cannot be extracted for s-pipes scripts with linux line endings, see
cz.cvut.spipes.util.QueryUtils.getQueryComment:The query string is split by
System.lineSeparator()which on windows is equal to\r\n(CRLF) but the query string contains linux line endings, i.e.\n(LF).Possible solution:
A possible solution would be to replace
System.lineSeparator()with"[\r\n]". This should work properly for windows, linux and mac line endings.Similar issues
There is one more place where
System.lineSeparator()is used and it may cause similar issues:cz.cvut.spipes.modules.TabularModule.executeSelfIssue cause
Issue found when debugging exception thrown by s-pipes. The exception was thrown when running s-pipes on windows with ava-model modules checked out with linux line endings.