Option -recreate is not working.

[dandaoatdell]

The 'recreate' option doesn't seem to work. To reproduce, create a stream 'my-stream' with some special setting. Examine the stream and not the setting. Execute the following command: /opt/pravega-benchmark/bin/pravega-benchmark -controller tcp://:9090 -stream my-stream -segments 1 -producers 1 -size 10 -recreate false -time 60 Examine the stream again, the setting is disappeared.

[Raghuveera-Rao]

Is this issue fixed ?

[RaulGracia]

@Raghuveera-Rao will check it.

[RaulGracia]

@dandaoatdell I have checked the -recreate option and it behaves as expected. This is the procedure I have used to verify it:

1) If we use -recreate True, we expect to have a new stream, which means that readers should not read any data from a previous execution on a Stream with the same name:

Experiment writing and reading 10 events from a Stream called my-stream

• Test 1:

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream -segments 1 -producers 1 -size 10 -recreate True -events 10
  2019-11-07 14:41:16:198 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Writing, 131.579 records/sec, 10 bytes record size, 0.00 MiB/sec, 12.4 ms avg latency, 69.0 ms max latency, 6 ms 50th, 6 ms 75th, 7 ms 95th, 7 ms 99th, 7 ms 99.9th, 7 ms 99.99th

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream -segments 1 -consumers 1 -size 10 -recreate True -time 10
  2019-11-07 14:41:33:180 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Reading, 0.990 records/sec, 10 bytes record size, 0.00 MiB/sec, 3.5 ms avg latency, 34.0 ms max latency, 0 ms 50th, 0 ms 75th, 1 ms 95th, 1 ms 99th, 1 ms 99.9th, 1 ms 99.99th.

• Test 2:

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream -segments 1 -producers 1 -size 10 -recreate True -events 10
  2019-11-07 14:46:00:473 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Writing, 121.951 records/sec, 10 bytes record size, 0.00 MiB/sec, 13.6 ms avg latency, 74.0 ms max latency, 7 ms 50th, 7 ms 75th, 8 ms 95th, 8 ms 99th, 8 ms 99.9th, 8 ms 99.99th.

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream -segments 1 -consumers 1 -size 10 -recreate True -time 10
  2019-11-07 14:46:15:247 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Reading, 0.990 records/sec, 10 bytes record size, 0.00 MiB/sec, 3.7 ms avg latency, 36.0 ms max latency, 0 ms 50th, 0 ms 75th, 1 ms 95th, 1 ms 99th, 1 ms 99.9th, 1 ms 99.99th.

Observations:

• In these test iterations, only 10 events have been read on the reader side, meaning that the Stream has been previously destroyed.
• An evidence of this can be seen in the logs writer side of the benchmark:

  2019-11-07 14:45:59:291 +0000 [grpc-default-executor-0] WARN io.pravega.client.stream.impl.ControllerImpl - [requestId=5231768222979732085] Stream already exists: my-stream Sealing and Deleteing the stream : my-stream and then recreating the same
  2019-11-07 14:45:59:970 +0000 [grpc-default-executor-0] INFO io.pravega.client.stream.impl.ControllerImpl - [requestId=1013985113166191641] Successfully deleted stream: my-stream
  2019-11-07 14:46:00:316 +0000 [grpc-default-executor-0] INFO io.pravega.client.stream.impl.ControllerImpl - [requestId=2877144219972877840] Stream created successfully: my-stream

2) If we use -recreate False, we expect to have the same Stream, which means that readers should read on each experiment an increasing number of events:

• Test 1:

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 1 -producers 1 -size 10 -recreate False -events 10
  2019-11-07 14:51:10:812 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Writing, 126.582 records/sec, 10 bytes record size, 0.00 MiB/sec, 12.2 ms avg latency, 71.0 ms max latency, 6 ms 50th, 6 ms 75th, 6 ms 95th, 6 ms 99th, 6 ms 99.9th, 6 ms 99.99th.

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 1 -consumers 1 -size 10 -recreate False -time 10
  2019-11-07 14:52:04:412 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Reading, 0.989 records/sec, 10 bytes record size, 0.00 MiB/sec, 5.2 ms avg latency, 51.0 ms max latency, 0 ms 50th, 0 ms 75th, 1 ms 95th, 1 ms 99th, 1 ms 99.9th, 1 ms 99.99th.

• Test 2:

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 1 -producers 1 -size 10 -recreate False -events 10
  2019-11-07 14:52:36:477 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Writing, 126.582 records/sec, 10 bytes record size, 0.00 MiB/sec, 12.9 ms avg latency, 71.0 ms max latency, 6 ms 50th, 7 ms 75th, 7 ms 95th, 7 ms 99th, 7 ms 99.9th, 7 ms 99.99th.

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 1 -consumers 1 -size 10 -recreate False -time 10
  2019-11-07 14:53:24:035 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 20 records Reading, 1.977 records/sec, 10 bytes record size, 0.00 MiB/sec, 2.7 ms avg latency, 53.0 ms max latency, 0 ms 50th, 0 ms 75th, 1 ms 95th, 1 ms 99th, 1 ms 99.9th, 1 ms 99.99th.

Observations:

• As you can see, readers read 10 events the first time and 20 events the second time. This means that the Stream has not been recreated, as specified in the write benchmark.

3) Maybe, what you @dandaoatdell have seen is the following: setting -recreate False and change the number of segments across executions. In this case, the Benchmark "scales" the Stream to start the workload with the desired number of Segments. This is an update of the Strem, not a recreation:

• Test 1

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 10 -producers 1 -size 10 -recreate False -events 10
  2019-11-07 14:57:01:641 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 10 records Writing, 88.496 records/sec, 10 bytes record size, 0.00 MiB/sec, 20.7 ms avg latency, 90.0 ms max latency, 5 ms 50th, 9 ms 75th, 13 ms 95th, 13 ms 99th, 13 ms 99.9th, 13 ms 99.99th.

  bin/pravega-benchmark -controller tcp://10.100.51.158:9090 -stream my-stream-2 -segments 10 -consumers 1 -size 10 -recreate False -time 10
  2019-11-07 14:58:53:833 +0000 [ForkJoinPool-1-worker-1] INFO io.pravega.perf.PerfStats - 30 records Reading, 2.941 records/sec, 10 bytes record size, 0.00 MiB/sec, 1.6 ms avg latency, 44.0 ms max latency, 0 ms 50th, 0 ms 75th, 1 ms 95th, 1 ms 99th, 1 ms 99.9th, 1 ms 99.99th.

Observations:

• Visibly, the -recreate False option worked, given that the reader read 30 events (20 from the previous test + 10 from the current one).
• The output of the Benchmark shows the process of manually scaling the Stream, which is part of the behavior of the tool:

  The stream: my-stream-2 will be manually scaling to 10 segments
  2019-11-07 14:57:00:179 +0000 [main] INFO io.pravega.client.admin.impl.StreamManagerImpl - Updating scope/stream: Scope/my-stream-2 with configuration: StreamConfiguration(scalingPolicy=ScalingPolicy(scaleType=FIXED_NUM_SEGMENTS, targetRate=0, scaleFactor=0, minNumSegments=10), retentionPolicy=null, timestampAggregationTimeout=0)

With these tests, I think that the - recreate option for Streams is working fine.

[Raghuveera-Rao]

Thanks @RaulGracia for the detailed explanation....