IBMStreams / streamsx.dps

The IBMStreams/streamsx.dps GitHub repository is home to the Streams toolkit named DPS (Distributed Process Store)
http://ibmstreams.github.io/streamsx.dps
Other
4 stars 9 forks source link

DpsWithReconnect Composites should forward punctuation #72

Closed Alex-Cook4 closed 7 years ago

Alex-Cook4 commented 7 years ago

DpsWithReconnect Composites should forward punctuation. Currently they don't handle punctuation.

brandtol commented 7 years ago

This problem also affects the final punctuations. They do not got through the composite, because the second input port (the metrics one). I would suggest to use

                onPunct In :
                {
                    submit( currentPunct() , Out);
                }

Or maybe we can get rid of the aggregate operator and do tumbling aggregations directly in the composite operator, to avoid the loopback.

Alex-Cook4 commented 7 years ago

@brandtol I agree with the first approach is good for handling both window/final punctuations.

I don't think I follow the issue with the aggregate operator (having sliding metrics were a key requirement). The docs state:

Sending final punctuation to an output port effectively closes the port. The operator discards all subsequent tuples and punctuation submitted to that port.

So if I understand correctly, we would submit a final punct to the output port of the operator/composite, thereby completing the stream. So in that case, the aggregate isn't an issue, right?

brandtol commented 7 years ago

In my test it worked that way. I am not sure if we also want to forward punctuations to the aggregate operator. For the final punct , it was not neccessary.