wso2 / product-sp

An open source, cloud-native streaming data integration and analytics product optimized for agile digital businesses
https://wso2.com/analytics-and-stream-processing/
Apache License 2.0
118 stars 132 forks source link

[EI Analytics] Invalid query generation for SQL Server #1032

Closed m-jahedbozorgan closed 3 years ago

m-jahedbozorgan commented 5 years ago

Description: Changed all datasources from H2 to Microsoft SQL Server. After calling one of services in ESB profile, the console for Dashboard Worker profile prints error log. Error log shows that an invalid query was executed because specifying "FETCH NEXT n ROWS ONLY" without specifying "OFFSET m ROWS" in not valid in Microsoft SQL Server. See attached error log file for more info.

C:\wso2ei-6.5.0>if "" == "" set JAVA_HOME=C:\WSO2EI~1.0\bin\..\jdk\jdk8u212-b03

C:\wso2ei-6.5.0>C:\WSO2EI~1.0\bin\analytics-worker.bat
"C:\WSO2EI~1.0\bin\..\wso2\analytics\bin\kernel-version.txt"
"C:\WSO2EI~1.0\wso2\ANALYT~1\bin\..\bin\version.txt"
JAVA_HOME environment variable is set to C:\WSO2EI~1.0\bin\..\jdk\jdk8u212-b03
CARBON_HOME environment variable is set to C:\WSO2EI~1.0\wso2\ANALYT~1\bin\..
RUNTIME_HOME environment variable is set to C:\WSO2EI~1.0\wso2\ANALYT~1\wso2\worker\bin\..
RUNTIME environment variable is set to worker
osgi> [2019-07-19 17:51:56,365]  INFO {org.wso2.msf4j.internal.websocket.WebSocketServerSC} - All required capabilities are available of WebSocket service component is available.
[2019-07-19 17:51:56,521]  INFO {org.wso2.carbon.metrics.core.config.model.JmxReporterConfig} - Creating JMX reporter for Metrics with domain 'org.wso2.carbon.metrics'
[2019-07-19 17:51:56,537]  INFO {org.wso2.msf4j.analytics.metrics.MetricsComponent} - Metrics Component is activated
[2019-07-19 17:51:56,537]  INFO {org.wso2.carbon.databridge.agent.internal.DataAgentDS} - Successfully deployed Agent Server
[2019-07-19 17:52:01,223]  INFO {org.wso2.carbon.cluster.coordinator.rdbms.internal.RDBMSCoordinationServiceComponent} - Cluster coordination has been disabled. Enable it in deployment.yaml to use the clustering service
[2019-07-19 17:52:01,223]  INFO {org.wso2.carbon.metrics.jdbc.core.config.model.JdbcReporterConfig} - Creating JDBC reporter for Metrics with source 'wso2-sp', data source 'java:comp/env/jdbc/WSO2MetricsDB' and 60 seconds polling period
[2019-07-19 17:52:01,239]  INFO {org.wso2.carbon.analytics.idp.client.core.utils.IdPServiceUtils} - IdP client of type 'local' is started.
[2019-07-19 17:52:01,301]  INFO {org.wso2.carbon.event.simulator.core.service.CSVFileDeployer} - CSV file deployer initiated.
[2019-07-19 17:52:01,301]  INFO {org.wso2.carbon.event.simulator.core.service.SimulationConfigDeployer} - Simulation config deployer initiated.
[2019-07-19 17:52:01,395]  INFO {org.wso2.msf4j.MicroservicesRunner} - Microservices server started in 78ms
[2019-07-19 17:52:01,395]  INFO {org.wso2.msf4j.MicroservicesRunner} - Microservices server started in 78ms
[2019-07-19 17:52:01,410]  INFO {org.wso2.transport.http.netty.contractimpl.listener.ServerConnectorBootstrap$HttpServerConnector} - HTTP(S) Interface starting on host 7443 and port 7443
[2019-07-19 17:52:01,410]  INFO {org.wso2.transport.http.netty.contractimpl.listener.ServerConnectorBootstrap$HttpServerConnector} - HTTP(S) Interface starting on host 7070 and port 7070
[2019-07-19 17:52:01,410]  INFO {org.wso2.carbon.stream.processor.statistics.api.StatisticsApi} - org.wso2.carbon.stream.processor.statistics.api.StatisticsApi service component has  started.
[2019-07-19 17:52:01,410]  INFO {org.wso2.carbon.stream.processor.statistics.internal.service.OperatingSystemMetricsServiceComponent} - OperatingSystemMetricsService Component activated
[2019-07-19 17:52:01,457]  INFO {org.wso2.carbon.databridge.receiver.binary.internal.BinaryDataReceiverServiceComponent} - org.wso2.carbon.databridge.receiver.binary.internal.Service Component is activated
[2019-07-19 17:52:01,457]  INFO {org.wso2.carbon.databridge.receiver.binary.internal.BinaryDataReceiver} - Started Binary SSL Transport on port : 9712
[2019-07-19 17:52:01,457]  INFO {org.wso2.carbon.databridge.receiver.binary.internal.BinaryDataReceiver} - Started Binary TCP Transport on port : 9612
[2019-07-19 17:52:01,457]  INFO {org.wso2.carbon.databridge.receiver.thrift.internal.ThriftDataReceiverDS} - Service Component is activated
[2019-07-19 17:52:01,473]  INFO {org.wso2.carbon.databridge.receiver.thrift.ThriftDataReceiver} - Thrift Server started at 0.0.0.0
[2019-07-19 17:52:01,473]  INFO {org.wso2.carbon.databridge.receiver.thrift.ThriftDataReceiver} - Thrift SSL port : 7712
[2019-07-19 17:52:01,489]  INFO {org.wso2.carbon.databridge.receiver.thrift.ThriftDataReceiver} - Thrift port : 7612
[2019-07-19 17:52:01,676]  INFO {org.wso2.msf4j.internal.MicroservicesServerSC} - All microservices are available
[2019-07-19 17:52:01,676]  INFO {org.wso2.transport.http.netty.contractimpl.listener.ServerConnectorBootstrap$HttpServerConnector} - HTTP(S) Interface starting on host 9091 and port 9091
[2019-07-19 17:52:01,676]  INFO {org.wso2.transport.http.netty.contractimpl.listener.ServerConnectorBootstrap$HttpServerConnector} - HTTP(S) Interface starting on host 9444 and port 9444
[2019-07-19 17:52:02,348]  INFO {org.quartz.impl.StdSchedulerFactory} - Using default implementation for ThreadExecutor
[2019-07-19 17:52:02,348]  INFO {org.quartz.simpl.SimpleThreadPool} - Job execution threads will use class loader of thread: Timer-0
[2019-07-19 17:52:02,364]  INFO {org.quartz.core.SchedulerSignalerImpl} - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
[2019-07-19 17:52:02,364]  INFO {org.quartz.core.QuartzScheduler} - Quartz Scheduler v.2.1.1 created.
[2019-07-19 17:52:02,364]  INFO {org.quartz.simpl.RAMJobStore} - RAMJobStore initialized.
[2019-07-19 17:52:02,364]  INFO {org.quartz.core.QuartzScheduler} - Scheduler meta-data: Quartz Scheduler (v2.1.1) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

[2019-07-19 17:52:02,364]  INFO {org.quartz.impl.StdSchedulerFactory} - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
[2019-07-19 17:52:02,364]  INFO {org.quartz.impl.StdSchedulerFactory} - Quartz scheduler version: 2.1.1
[2019-07-19 17:52:02,364]  INFO {org.quartz.core.QuartzScheduler} - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
[2019-07-19 17:52:02,645]  INFO {org.wso2.siddhi.core.aggregation.IncrementalDataPurging} - Data purging has enabled for tables: SECONDS,MINUTES,HOURS,DAYS,MONTHS,YEARS, with an interval of 3600 seconds in ESBStatAgg aggregation
[2019-07-19 17:52:02,645]  INFO {org.wso2.siddhi.core.aggregation.IncrementalDataPurging} - Data purging has enabled for tables: SECONDS,MINUTES,HOURS,DAYS,MONTHS,YEARS, with an interval of 3600 seconds in MediatorStatAgg aggregation
[2019-07-19 17:52:02,645]  INFO {org.wso2.carbon.stream.processor.core.internal.StreamProcessorService} - Siddhi App EI_Analytics_StatApp deployed successfully
[2019-07-19 17:52:02,645]  INFO {org.wso2.carbon.kernel.internal.CarbonStartupHandler} - WSO2 Stream Processor started in 8.639 sec
[2019-07-19 17:54:56,465]  INFO {org.wso2.carbon.databridge.core.DataBridge} - user admin connected
[2019-07-19 17:56:21,557] ERROR {org.wso2.siddhi.core.stream.StreamJunction} - Error in 'EI_Analytics_StatApp' after consuming events from Stream 'PreProcessedESBStatStream', Error executing 'store_select_query_ESBStatAgg_YEARS', Error when preparing to execute query: 'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.SHARD_ID AS SHARD_ID, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS WHERE (ESBStatAgg_YEARS.SHARD_ID = ? ) ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'ESBStatAgg_YEARS'. Hence, dropping event 'StreamEvent{ timestamp=1563542781358, beforeWindowData=null, onAfterWindowData=[0], outputData=[NotifyCustomsDeclarationStatusChange@0:NotifyCustomsDeclarationStatusChange, NotifyCustomsDeclarationStatusChange, proxy service, 78, 0, 1563542779486, NotifyCustomsDeclarationStatusChange, -1234], type=CURRENT, next=null}' org.wso2.siddhi.core.exception.StoreQueryRuntimeException: Error executing 'store_select_query_ESBStatAgg_YEARS', Error when preparing to execute query: 'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.SHARD_ID AS SHARD_ID, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS WHERE (ESBStatAgg_YEARS.SHARD_ID = ? ) ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'ESBStatAgg_YEARS'
        at org.wso2.siddhi.core.query.SelectStoreQueryRuntime.execute(SelectStoreQueryRuntime.java:68)
        at org.wso2.siddhi.core.aggregation.RecreateInMemoryData.recreateInMemoryData(RecreateInMemoryData.java:123)
        at org.wso2.siddhi.core.aggregation.AggregationRuntime.recreateInMemoryData(AggregationRuntime.java:365)
        at org.wso2.siddhi.core.aggregation.IncrementalAggregationProcessor.process(IncrementalAggregationProcessor.java:77)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:122)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:181)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:405)
        at org.wso2.siddhi.core.query.output.callback.InsertIntoStreamCallback.send(InsertIntoStreamCallback.java:56)
        at org.wso2.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:97)
        at org.wso2.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:46)
        at org.wso2.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
        at org.wso2.siddhi.core.query.processor.filter.FilterProcessor.process(FilterProcessor.java:65)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:122)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:181)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:405)
        at org.wso2.siddhi.core.query.output.callback.InsertIntoStreamCallback.send(InsertIntoStreamCallback.java:56)
        at org.wso2.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:97)
        at org.wso2.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:46)
        at org.wso2.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
        at org.wso2.extension.siddhi.execution.esbanalytics.decompress.DecompressStreamProcessorExtension.process(DecompressStreamProcessorExtension.java:260)
        at org.wso2.siddhi.core.query.processor.stream.StreamProcessor.processEventChunk(StreamProcessor.java:47)
        at org.wso2.siddhi.core.query.processor.stream.AbstractStreamProcessor.process(AbstractStreamProcessor.java:123)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:133)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:204)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:414)
        at org.wso2.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
        at org.wso2.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:44)
        at org.wso2.siddhi.core.stream.input.InputHandler.send(InputHandler.java:73)
        at org.wso2.siddhi.core.stream.input.source.PassThroughSourceHandler.sendEvent(PassThroughSourceHandler.java:35)
        at org.wso2.siddhi.core.stream.input.source.InputEventHandler.sendEvent(InputEventHandler.java:81)
        at org.wso2.extension.siddhi.map.wso2event.source.WSO2SourceMapper.mapAndProcess(WSO2SourceMapper.java:227)
        at org.wso2.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:149)
        at org.wso2.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:115)
        at org.wso2.extension.siddhi.io.wso2event.source.AgentCallbackImpl.receive(AgentCallbackImpl.java:72)
        at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:88)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.extension.siddhi.store.rdbms.exception.RDBMSTableException: Error when preparing to execute query: 'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.SHARD_ID AS SHARD_ID, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS WHERE (ESBStatAgg_YEARS.SHARD_ID = ? ) ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'ESBStatAgg_YEARS'
        at org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.query(RDBMSEventTable.java:1767)
        at org.wso2.siddhi.core.table.record.AbstractQueryableRecordTable.query(AbstractQueryableRecordTable.java:90)
        at org.wso2.siddhi.core.query.SelectStoreQueryRuntime.execute(SelectStoreQueryRuntime.java:55)
        ... 45 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid usage of the option NEXT in the FETCH statement.
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:592)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:522)
        at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2935)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:444)
        at com.zaxxer.hikari.proxy.PreparedStatementProxy.executeQuery(PreparedStatementProxy.java:52)
        at com.zaxxer.hikari.proxy.HikariPreparedStatementProxy.executeQuery(HikariPreparedStatementProxy.java)
        at org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.query(RDBMSEventTable.java:1750)
        ... 47 more

[2019-07-19 17:56:21,573] ERROR {org.wso2.siddhi.core.stream.StreamJunction} - Error in 'EI_Analytics_StatApp' after consuming events from Stream 'PreProcessedMediatorStatStream', Error executing 'store_select_query_MediatorStatAgg_YEARS', Error when preparing to execute query: 'SELECT MediatorStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, MediatorStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.metaTenantId AS metaTenantId, MediatorStatAgg_YEARS.componentId AS componentId, MediatorStatAgg_YEARS.componentName AS componentName, MediatorStatAgg_YEARS.componentType AS componentType, MediatorStatAgg_YEARS.entryPoint AS entryPoint, MediatorStatAgg_YEARS.entryPointHashcode AS entryPointHashcode, MediatorStatAgg_YEARS.hashCode AS hashCode, MediatorStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.startTime AS startTime, MediatorStatAgg_YEARS.SHARD_ID AS SHARD_ID, MediatorStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, MediatorStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, MediatorStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, MediatorStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, MediatorStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM MediatorStatAgg_YEARS WHERE (MediatorStatAgg_YEARS.SHARD_ID = ? ) ORDER BY MediatorStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'MediatorStatAgg_YEARS'. Hence, dropping event 'StreamEvent{ timestamp=1563542781358, beforeWindowData=null, onAfterWindowData=[0], outputData=[-1234, NotifyCustomsDeclarationStatusChange, -1496329460, NotifyCustomsDeclarationStatusChange@0:NotifyCustomsDeclarationStatusChange, -1496329460, NotifyCustomsDeclarationStatusChange, proxy service, 78, 0, 1563542779486], type=CURRENT, next=StreamEvent{ timestamp=1563542781358, beforeWindowData=null, onAfterWindowData=[0], outputData=[-1234, NotifyCustomsDeclarationStatusChange, -1496329460, NotifyCustomsDeclarationStatusChange@1:AnonymousEndpoint, 617252700, AnonymousEndpoint, endpoint, 78, 0, 1563542779486], type=CURRENT, next=StreamEvent{ timestamp=1563542781358, beforeWindowData=null, onAfterWindowData=[0], outputData=[-1234, NotifyCustomsDeclarationStatusChange, -1496329460, NotifyCustomsDeclarationStatusChange@2:PROXY_OUTSEQ, 442587239, PROXY_OUTSEQ, sequence, 0, 0, 1563542779564], type=CURRENT, next=StreamEvent{ timestamp=1563542781358, beforeWindowData=null, onAfterWindowData=[0], outputData=[-1234, NotifyCustomsDeclarationStatusChange, -1496329460, NotifyCustomsDeclarationStatusChange@3:SendMediator, 741641055, SendMediator, mediator, 0, 0, 1563542779564], type=CURRENT, next=null}}}}' org.wso2.siddhi.core.exception.StoreQueryRuntimeException: Error executing 'store_select_query_MediatorStatAgg_YEARS', Error when preparing to execute query: 'SELECT MediatorStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, MediatorStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.metaTenantId AS metaTenantId, MediatorStatAgg_YEARS.componentId AS componentId, MediatorStatAgg_YEARS.componentName AS componentName, MediatorStatAgg_YEARS.componentType AS componentType, MediatorStatAgg_YEARS.entryPoint AS entryPoint, MediatorStatAgg_YEARS.entryPointHashcode AS entryPointHashcode, MediatorStatAgg_YEARS.hashCode AS hashCode, MediatorStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.startTime AS startTime, MediatorStatAgg_YEARS.SHARD_ID AS SHARD_ID, MediatorStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, MediatorStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, MediatorStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, MediatorStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, MediatorStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM MediatorStatAgg_YEARS WHERE (MediatorStatAgg_YEARS.SHARD_ID = ? ) ORDER BY MediatorStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'MediatorStatAgg_YEARS'
        at org.wso2.siddhi.core.query.SelectStoreQueryRuntime.execute(SelectStoreQueryRuntime.java:68)
        at org.wso2.siddhi.core.aggregation.RecreateInMemoryData.recreateInMemoryData(RecreateInMemoryData.java:123)
        at org.wso2.siddhi.core.aggregation.AggregationRuntime.recreateInMemoryData(AggregationRuntime.java:365)
        at org.wso2.siddhi.core.aggregation.IncrementalAggregationProcessor.process(IncrementalAggregationProcessor.java:77)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:122)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:181)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:405)
        at org.wso2.siddhi.core.query.output.callback.InsertIntoStreamCallback.send(InsertIntoStreamCallback.java:56)
        at org.wso2.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:97)
        at org.wso2.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:46)
        at org.wso2.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:122)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:181)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:405)
        at org.wso2.siddhi.core.query.output.callback.InsertIntoStreamCallback.send(InsertIntoStreamCallback.java:56)
        at org.wso2.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:97)
        at org.wso2.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:46)
        at org.wso2.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
        at org.wso2.extension.siddhi.execution.esbanalytics.decompress.DecompressStreamProcessorExtension.process(DecompressStreamProcessorExtension.java:260)
        at org.wso2.siddhi.core.query.processor.stream.StreamProcessor.processEventChunk(StreamProcessor.java:47)
        at org.wso2.siddhi.core.query.processor.stream.AbstractStreamProcessor.process(AbstractStreamProcessor.java:123)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:187)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:97)
        at org.wso2.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:133)
        at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:204)
        at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:414)
        at org.wso2.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
        at org.wso2.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:44)
        at org.wso2.siddhi.core.stream.input.InputHandler.send(InputHandler.java:73)
        at org.wso2.siddhi.core.stream.input.source.PassThroughSourceHandler.sendEvent(PassThroughSourceHandler.java:35)
        at org.wso2.siddhi.core.stream.input.source.InputEventHandler.sendEvent(InputEventHandler.java:81)
        at org.wso2.extension.siddhi.map.wso2event.source.WSO2SourceMapper.mapAndProcess(WSO2SourceMapper.java:227)
        at org.wso2.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:149)
        at org.wso2.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:115)
        at org.wso2.extension.siddhi.io.wso2event.source.AgentCallbackImpl.receive(AgentCallbackImpl.java:72)
        at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:88)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.extension.siddhi.store.rdbms.exception.RDBMSTableException: Error when preparing to execute query: 'SELECT MediatorStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, MediatorStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.metaTenantId AS metaTenantId, MediatorStatAgg_YEARS.componentId AS componentId, MediatorStatAgg_YEARS.componentName AS componentName, MediatorStatAgg_YEARS.componentType AS componentType, MediatorStatAgg_YEARS.entryPoint AS entryPoint, MediatorStatAgg_YEARS.entryPointHashcode AS entryPointHashcode, MediatorStatAgg_YEARS.hashCode AS hashCode, MediatorStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, MediatorStatAgg_YEARS.startTime AS startTime, MediatorStatAgg_YEARS.SHARD_ID AS SHARD_ID, MediatorStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, MediatorStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, MediatorStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, MediatorStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, MediatorStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM MediatorStatAgg_YEARS WHERE (MediatorStatAgg_YEARS.SHARD_ID = ? ) ORDER BY MediatorStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY' on 'MediatorStatAgg_YEARS'
        at org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.query(RDBMSEventTable.java:1767)
        at org.wso2.siddhi.core.table.record.AbstractQueryableRecordTable.query(AbstractQueryableRecordTable.java:90)
        at org.wso2.siddhi.core.query.SelectStoreQueryRuntime.execute(SelectStoreQueryRuntime.java:55)
        ... 44 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid usage of the option NEXT in the FETCH statement.
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:592)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:522)
        at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2935)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:444)
        at com.zaxxer.hikari.proxy.PreparedStatementProxy.executeQuery(PreparedStatementProxy.java:52)
        at com.zaxxer.hikari.proxy.HikariPreparedStatementProxy.executeQuery(HikariPreparedStatementProxy.java)
        at org.wso2.extension.siddhi.store.rdbms.RDBMSEventTable.query(RDBMSEventTable.java:1750)
        ... 46 more

Suggested Labels: Type/Bug,Priority/High,Severity/Blocker

Suggested Assignees:

Affected Product Version: SP 4.4.0

OS, DB, other environment details and versions:
Environment is WSO2 EI (Integrator and Dashboard Worker profiles) on Windows + Microsoft SQL Server 2017

Steps to reproduce:

  1. Create these 6 empty databases in Microsoft SQL Server: WSO2_DASHBOARD_DB WSO2_PERMISSIONS_DB WSO2_EI_ANALYTICS_DB WSO2_CLUSTER_DB WSO2_PERSISTENCE_DB WSO2_METRICS_DB
  2. Run [EI_HOME]\wso2\analytics\wso2\dashboard\dbscripts\metrics\mssql.sql file on WSO2_METRICS_DB.
  3. Grant db_owner permissions to wso2 user on all above databases
  4. Update datasources in [EI_HOME]\wso2\analytics\conf\dashboard\deployment.yaml (WSO2_DASHBOARD_DB, EI_ANALYTICS, and WSO2_PERMISSIONS_DB)
  5. Update datasources in [EI_HOME]\wso2\analytics\conf\worker\deployment.yaml (WSO2_CLUSTER_DB, EI_ANALYTICS)
  6. Download and copy the 'Microsoft JDBC Driver For SQL Server' driver and its dependency 'OSGi Service JDBC' (2 jar files) to the \wso2\analytics\lib\ directory.
  7. Enable publishing ESB data to the Analytics profile by updating "Configuration to enable mediation flow analytics" section of \conf\synapse.properties file
  8. Run Analytics worker profile, then run Integrator profile
  9. Call one of ESB proxy services

Related Issues:

error analytics mssql.txt

m-jahedbozorgan commented 5 years ago

The same error occurs when I open the dashboard portal and want to view stats after manually copying contents of all tables in my H2 EI_ANALYTICS database to tables in my Microsoft SQL Server database and updating datasources.

Using Profiler I got executed queries when viewing my dashboard. These are executed queries:

exec sp_executesql N'SELECT URL, OWNER, NAME, DESCRIPTION, PARENT_ID, LANDING_PAGE, CONTENT FROM DASHBOARD_RESOURCE WHERE URL = @P0        ',N'@P0 varchar(8000)','eianalytics'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsStatsChart'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsStatsChart'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsHorizontalBarChart'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsTPS'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','Overall-Message-Count'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsHorizontalBarChart'
--exec sp_executesql N'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsHorizontalBarChart'
--exec sp_executesql N'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsHorizontalBarChart'
--exec sp_executesql N'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY'
exec sp_executesql N'SELECT WIDGET_NAME, WIDGET_CONFIGS FROM WIDGET_RESOURCE WHERE WIDGET_ID = @P0        ',N'@P0 varchar(8000)','EIAnalyticsHorizontalBarChart'
--exec sp_executesql N'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY'
--exec sp_executesql N'SELECT ESBStatAgg_YEARS.AGG_TIMESTAMP AS AGG_TIMESTAMP, ESBStatAgg_YEARS.AGG_EVENT_TIMESTAMP AS AGG_EVENT_TIMESTAMP, ESBStatAgg_YEARS.metaTenantId AS metaTenantId, ESBStatAgg_YEARS.componentId AS componentId, ESBStatAgg_YEARS.componentName AS componentName, ESBStatAgg_YEARS.componentType AS componentType, ESBStatAgg_YEARS.entryPoint AS entryPoint, ESBStatAgg_YEARS.AGG_LAST_EVENT_TIMESTAMP AS AGG_LAST_EVENT_TIMESTAMP, ESBStatAgg_YEARS.eventTimestamp AS eventTimestamp, ESBStatAgg_YEARS.AGG_SUM_duration AS AGG_SUM_duration, ESBStatAgg_YEARS.AGG_COUNT AS AGG_COUNT, ESBStatAgg_YEARS.AGG_MIN_duration AS AGG_MIN_duration, ESBStatAgg_YEARS.AGG_MAX_duration AS AGG_MAX_duration, ESBStatAgg_YEARS.AGG_SUM_faultCount AS AGG_SUM_faultCount FROM ESBStatAgg_YEARS ORDER BY ESBStatAgg_YEARS.AGG_TIMESTAMP DESC FETCH NEXT 1 ROWS ONLY'

Commented SELECT statements are invalid in Microsoft SQL Server because FETCH NEXT n ROWS ONLY must be used with OFFSET m ROWS.

RAVEENSR commented 4 years ago

Related issue in Siddhi-Store-RDBMS is https://github.com/siddhi-io/siddhi-store-rdbms/issues/156 and this issue is fixed in the latest siddhi-store-rdbms versions both in 4.xx and 5.xx versions.