search

spring-cloud / spring-cloud-deployer-local

The Spring Cloud Deployer implementation for a "local" machine
Apache License 2.0
39 stars 53 forks source link

Missing proper instance index key #114

Closed jvalkeal closed 6 years ago

jvalkeal commented 6 years ago

See spring-cloud/spring-cloud-dataflow#2164

jvalkeal commented 6 years ago

Looks like we're forced to set INSTANCE_INDEX is all cases as plain spring.cloud.stream.instanceIndex doesn't work.

jvalkeal commented 6 years ago

Actually adding INSTANCE_INDEX to SAJ works when I use only one producer. 

dataflow:>stream undeploy --name foostream
dataflow:>stream deploy --name foostream --properties "deployer.time.count=1,deployer.log.count=2"
partition 0
2018-04-26 18:41:35.673  INFO 6774 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:35
2018-04-26 18:41:37.659  INFO 6774 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:37
2018-04-26 18:41:39.661  INFO 6774 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:39
2018-04-26 18:41:41.662  INFO 6774 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:41

partition 1
2018-04-26 18:41:36.669  INFO 6775 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:36
2018-04-26 18:41:38.660  INFO 6775 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:38
2018-04-26 18:41:40.661  INFO 6775 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:40
2018-04-26 18:41:42.663  INFO 6775 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:41:42

Originally I thought that INSTANCE_INDEX from SAJ is totally skipped when I used two producers.

dataflow:>stream deploy --name foostream --properties "deployer.*.count=2"

First run somewhat mixes content.

partition 0
2018-04-26 18:46:38.069  INFO 8987 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:38
2018-04-26 18:46:39.058  INFO 8987 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:39
2018-04-26 18:46:40.059  INFO 8987 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:40

partition 1
2018-04-26 18:46:37.056  INFO 8988 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:37
2018-04-26 18:46:37.824  INFO 8988 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:37
2018-04-26 18:46:38.825  INFO 8988 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:46:38

Second run fully spreads content.

partition 0
2018-04-26 18:48:05.048  INFO 9451 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:05
2018-04-26 18:48:06.035  INFO 9451 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:06
2018-04-26 18:48:07.037  INFO 9451 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:07
2018-04-26 18:48:08.038  INFO 9451 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:08
partition 1
2018-04-26 18:48:04.067  INFO 9453 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:03
2018-04-26 18:48:04.751  INFO 9453 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:04
2018-04-26 18:48:05.751  INFO 9453 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:05
2018-04-26 18:48:06.753  INFO 9453 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:06
2018-04-26 18:48:07.754  INFO 9453 --- [ime.foostream-1] log-sink                                 : 04/26/18 18:48:07

I don't remember it anymore if this was a case with rabbit that partitioning doesn't exactly work.