search

thingsboard / thingsboard-edge

Apache License 2.0
98 stars 74 forks source link

[Bug] edge error log #60

Closed ryan-2048 closed 4 months ago

ryan-2048 commented 1 year ago

ThingsBoard Edge 3.5.1 Release ThingsBoard 3.5.1 Release Ubuntu 20.04 docker compose

This error is repeated

2023-06-09 06:13:02,207 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,209 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,319 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,321 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,415 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,417 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,488 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,490 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,558 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,560 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,627 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,630 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,703 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,706 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,777 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,779 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,846 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,849 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,916 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,919 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:02,991 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:02,994 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,064 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,067 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,150 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,152 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,222 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,225 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,293 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,296 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,365 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,367 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,439 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,442 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,513 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,515 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,589 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,591 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,661 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,663 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,731 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,733 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,832 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,835 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,907 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,909 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:03,980 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:03,983 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,035 [sql-log-1-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - Queue-0 [Cloud Events] queueSize [0] totalAdded [114] totalSaved [114] totalFailed [0]
2023-06-09 06:13:04,051 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,053 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,124 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,127 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,196 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,199 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,269 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,271 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,344 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,348 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,424 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,426 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,506 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,510 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,589 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,592 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,661 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,664 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,732 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,734 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,760 [sql-log-1-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - Queue-0 [Attributes] queueSize [0] totalAdded [507] totalSaved [507] totalFailed [0]
2023-06-09 06:13:04,761 [sql-log-1-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - Queue-1 [Attributes] queueSize [0] totalAdded [395] totalSaved [395] totalFailed [0]
2023-06-09 06:13:04,762 [sql-log-1-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - Queue-2 [Attributes] queueSize [0] totalAdded [79] totalSaved [79] totalFailed [0]
2023-06-09 06:13:04,805 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,807 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,874 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,877 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:04,951 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:04,954 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,026 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,029 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,098 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,100 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,170 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,172 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,238 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,240 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,315 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,317 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,389 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,391 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,470 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,473 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,543 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,546 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,631 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,634 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,706 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,709 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,777 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,780 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,849 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,851 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:05,923 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:05,925 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:06,003 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:06,006 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:06,090 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:06,092 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:06,171 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:06,180 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-09 06:13:06,323 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-09 06:13:06,325 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
volodymyr-babak commented 1 year ago

Hello @Kevin-ox,

I appreciate your patience and cooperation in this matter. Could you kindly confirm when you installed TB Edge 3.5.1? Was it installed recently, or had it been in place for some time? We released a version on June 1st which unfortunately contained a bug, but we quickly rectified this issue and re-released an updated version on July 2nd.

From your logs, it seems like the error message corresponds to the bug we fixed in the July 2nd update. Hence, I'm wondering if you might have downloaded the bug-containing version on July 1st, and are currently using that.

Kindly let me know so we can better assist you in resolving this issue.

ryan-2048 commented 1 year ago

yesterday,I use source-compiled docker: https://github.com/thingsboard/thingsboard-edge/archive/refs/tags/v3.5.1.zip

volodymyr-babak commented 1 year ago

Thank you for your prompt response - in this case it should be the latest version. To assist you better, could you kindly provide a detailed step-by-step guide on how to reproduce the issue you are experiencing?

While it is currently unclear as to how this particular situation might arise under normal operation, it's worth noting that typically, the array size must be at least '1' to avoid such errors.

ryan-2048 commented 1 year ago

And here's another really weird thing

WechatIMG1328
ryan-2048 commented 1 year ago

I originally had a cloud with ThingsBoard 3.4.4 Release, I upgraded the cloud to 3.5.1 and then reinstalled the edge

volodymyr-babak commented 1 year ago

Thank you for additional information. An excessive number of cloud events does indeed indicate a potential bug within Edge. To better understand and diagnose the issue, we would greatly appreciate if you could provide detailed steps to reproduce the problem.

As an example, these steps might look like:

If you could follow this structure and include any additional steps or specific details (for instance, the type of device being assigned, any unique configurations, etc.), this will allow us to replicate the situation on our end and expedite the resolution process.

We appreciate your assistance in this matter.

ryan-2048 commented 1 year ago

Too many cloud events will also cause the attribute update of the cloud to be in the pending state and not delivered. Is there any way I can set the ttl of the cloud event

ryan-2048 commented 1 year ago

Too many cloud events will also cause the attribute update of the cloud to be in the pending state and not delivered. Is there any way I can set the ttl of the cloud event

Because the edge is disconnected and not automatically connected

volodymyr-babak commented 1 year ago

You can adjust the settings for cloud events by updating the tb-edge.conf file. You can find more information on this here: https://thingsboard.io/docs/pe/edge/user-guide/install/config/#sql-configuration-parameters

Below is the list of environment variables that you can modify to set the Time To Live (TTL) for cloud events:

SQL_TTL_CLOUD_EVENTS_ENABLED: This variable enables or disables the TTL for cloud events. SQL_TTL_CLOUD_EVENTS_EXECUTION_INTERVAL: This variable determines how often the system should check for expired events. SQL_TTL_CLOUD_EVENTS_TTL: This variable sets the time period after which cloud events are considered expired.

Please make sure to carefully adjust these settings to ensure the system behaves as expected.

ryan-2048 commented 1 year ago

Thank you for your reply. Now I have encountered a new and important problem, the link between the edge and the cloud is not normal, the active of the edge is false on the cloud, all attribute updates are pending, but the telemetry and attribute upload are normal, and can be restored by restarting the edge or the cloud. I am trying to troubleshoot the error.

ryan-2048 commented 1 year ago

The link problem is located, please see the log: Non-critical logs have been omitted, and I think the reason is that the edge is reconnecting to the cloud, but the edge does not know that it is disconnected from the cloud. Log of the cloud:

...
2023-06-12 05:07:08,999 [grpc-default-executor-7] INFO  o.t.s.s.edge.rpc.EdgeGrpcService - [75c0aa86-d355-450b-9a57-8e296427da7b] edge [92472310-f6c1-11ed-9128-87aaac9fd8c2] connected successfully.
2023-06-12 05:07:09,446 [grpc-default-executor-8] ERROR o.t.s.s.edge.rpc.EdgeGrpcSession - [b151534e-4cd7-4797-950b-b4c6e2924eeb] Stream was terminated due to error:
io.grpc.StatusRuntimeException: CANCELLED: client cancelled
    at io.grpc.Status.asRuntimeException(Status.java:526)
    at io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onCancel(ServerCalls.java:291)
    at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.closedInternal(ServerCallImpl.java:362)
    at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.closed(ServerCallImpl.java:350)
    at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1Closed.runInContext(ServerImpl.java:913)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-06-12 05:07:09,446 [grpc-default-executor-8] INFO  o.t.s.s.edge.rpc.EdgeGrpcService - [92472310-f6c1-11ed-9128-87aaac9fd8c2] edge disconnected!
...

Edge of the log:

2023-06-12 05:07:05,776 [grpc-default-executor-1] WARN  o.t.edge.rpc.EdgeGrpcClient - [1d288d4e-69da-e984-1bc1-6fe9a42a82e3] Stream was terminated due to error:
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
    at io.grpc.Status.asRuntimeException(Status.java:535)
    at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
    at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
    at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
    at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection timed out
2023-06-12 05:07:08,860 [cloud-manager-reconnect-58-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Trying to reconnect due to the error: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception!
2023-06-12 05:07:08,870 [rule-dispatcher-3-3] INFO  o.t.rule.engine.node.agc.AgcNode - abEntry_userPower value= 16.557205200195312
2023-06-12 05:07:08,871 [cloud-manager-reconnect-58-thread-1] INFO  o.t.edge.rpc.EdgeGrpcClient - [1d288d4e-69da-e984-1bc1-6fe9a42a82e3] Sending a connect request to the TB!
2023-06-12 05:07:08,874 [rule-dispatcher-3-3] INFO  o.t.rule.engine.node.agc.AgcNode - abEntry_userPower value= 16.557205200195312
2023-06-12 05:07:09,023 [grpc-default-executor-0] INFO  o.t.edge.rpc.EdgeGrpcClient - [1d288d4e-69da-e984-1bc1-6fe9a42a82e3] Configuration received: edgeIdMSB: -7906312065059319315
edgeIdLSB: -7986984772031227710
tenantIdMSB: 8562029609502118381
tenantIdLSB: -7986984772031227710
customerIdMSB: 5107175096534438381
customerIdLSB: -4663170113026818111
...
volodymyr-babak commented 1 year ago

Hello @Kevin-ox,

Thank you for this insightful update. It appears that we might be dealing with a potentially new bug that's been introduced alongside the decreased ping timeouts for the edge-cloud gRPC connection.

To better understand this issue, could you kindly provide the full logs for both the cloud and edge services? This will greatly assist us in performing a more in-depth analysis.

volodymyr-babak commented 1 year ago

Hello @Kevin-ox,

I'm wondering if you're also building your cloud from sources? I recently patched a bug that was causing instability in the connections between edge and cloud. In some instances, the edge appeared to be connected to the cloud, but the cloud was not aware of it due to an incorrect session being removed from the list, thereby making the edge disconnected on the cloud. You can find more details here: https://github.com/thingsboard/thingsboard/pull/8727/files

I'm suspecting that you might be experiencing a similar issue. Would it be possible for you to compile the cloud from sources and test it within your setup to verify this possibility? Your assistance in this would be immensely valuable.

Thanks for your continued cooperation.

ryan-2048 commented 1 year ago

log.zip This is log file, I will try to compile from source cloud. Another problem at the edge:

2023-06-12 00:00:00,165 [cloud-manager-57-thread-1] INFO  o.t.s.s.cloud.CloudManagerService - Resetting seqIdOffset - new cycle started
2023-06-12 00:00:00,168 [cloud-manager-57-thread-1] WARN  o.t.s.s.cloud.CloudManagerService - Failed to process messages handling!
java.lang.IndexOutOfBoundsException: Index: -1
    at java.base/java.util.Collections$EmptyList.get(Collections.java:4483)
    at org.thingsboard.server.service.cloud.CloudManagerService.lambda$processHandleMessages$1(CloudManagerService.java:273)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

I solved it and will contribute to the edge

ryan-2048 commented 1 year ago

Hello @volodymyr-babak I built the cloud from source and ran it for a while, then wrote a script to update the properties to the edge once every two seconds, and the problem still occurred, but the difference this time was that the active attribute of the edge was true, the cloud property update was pending down, and the uplink data was still normal. The difference between my log time and the picture time is 8 hours, that is, the problem occurred at 03:35:35 in the log

attributes down attributes

log.zip

volodymyr-babak commented 1 year ago

Hello @Kevin-ox,

Thanks for providing the logs. From what I can see, for some reason, uplink messages from the edge to the cloud are not being delivered: 2023-06-13 03:35:40,721 [cloud-manager-57-thread-1] WARN o.t.s.s.cloud.CloudManagerService - Failed to deliver the batch: [uplinkMsgId: 774458914

I don't notice any cloud-related errors. This situation appears to be some sort of communication issue. Can you please reduce the number of messages that are sent from the edge to the cloud, perhaps set it to 1 or 5? Here's a link to the relevant documentation: https://thingsboard.io/docs/edge/user-guide/install/config/#cloud-parameters

Please update the following parameter, CLOUD_RPC_STORAGE_MAX_READ_RECORDS_COUNT, and set it to 1 or 5 in your tb-edge.conf file: export CLOUD_RPC_STORAGE_MAX_READ_RECORDS_COUNT=1

Please let me know if this adjustment helped alleviate the communication issue.

ryan-2048 commented 1 year ago

Hello @volodymyr-babak It looks like the uplink message is not being delivered, but if the active of the edge is true, the downlink is all in the pending state. In the picture I just showed, there are 50 messages in the pending state (then the active of the edge is true).

I will continue to test it, and I will give feedback if there are any questions. Thank you.

volodymyr-babak commented 1 year ago

@Kevin-ox

We have identified the root cause of the issue mentioned in this ticket. Given the critical nature of this issue, which could potentially lead to data loss, we have decided to release a hotfix version.

The hotfix for the Community Edition (CE 3.5.1.1) has been completed and released. It can be accessed at the following link: https://github.com/thingsboard/thingsboard-edge/releases/tag/v3.5.1.1

For reference, the specific commit that rectifies the IndexOutOfBoundException issue can be viewed here: https://github.com/thingsboard/thingsboard-edge/commit/db947eccd63e4b9d498d37c213c95d9f73a2124c

Please note, we are also in the process of preparing a hotfix for the Professional Edition (PE 3.5.1.1), which will be released shortly.

volodymyr-babak commented 1 year ago

The Professional Edition (PE 3.5.1.1) hotfix has also been released. Please follow the upgrade guide to update your ThingsBoard Edge instances. It's worth noting that this update doesn't require a database update, only a package update, so it should be a quick process.

For the Community Edition (CE) upgrade instructions, follow this link: https://thingsboard.io/docs/user-guide/install/edge/upgrade-instructions/#upgrading-to-3511

For the Professional Edition (PE) upgrade instructions, refer to this link: https://thingsboard.io/docs/user-guide/install/pe/edge/upgrade-instructions/#upgrading-to-3511

Should you encounter any issues after the update, please don't hesitate to inform me. I apologize for any inconveniences this bug may have caused.

ryan-2048 commented 1 year ago

That's great! thank you.