Proposal: streamsx.beam

[mrshenli]

Proposal

This is a proposal for creating a repo to publish Beam samples for IBM Streams. These samples can demonstrate how to connect with IBM Cloud services using Beam, as well as, how to make use of IBM Streams Beam runner specific features.

Motivation

Currently, Beam samples are published in IBM Streams Beam runner releases. A dedicated public repo will help us to get samples to potential clients faster, independently from runner release cycles.

Sample Categories

The samples will include but not limited to the following categories:

• Launching Beam applications in different contexts (BUNDLE, DISTRIBUTED, and STREAMING_ANALYTICS_SERVICE)
• Use Beam applications to access various IBM Cloud services (e.g., MessageHub, COS, etc.)
• Use Beam metrics to monitor Streams application status
• IBM Streams Beam runner specific performance tuning options
• Publish or subscribe to tuple streams in other Beam or Streams applications

[pgerv12]

Perhaps we can put this in a Beam directory under the Samples repo?

[mrshenli]

@pgerv12 regarding whether it should be a repo or a folder in IBMStreams/samples, my vote would be creating a new repo. Because some of the samples are engine-independent, e.g., using the S3 connector to write to IBM COS.

[ddebrunner]

If it is a seprate repo then we need to ensure that the samples still can appear here:

http://ibmstreams.github.io/samples/

[ddebrunner]

+1 to the concept.

[mikespicer]

+1 to concept and Dan's comment about making sure that any samples in the repo also show up in http://ibmstreams.github.io/samples/

[natashadsilva]

I'm not convinced a separate repo is going to make the samples easier to find than having a Beam folder within the samples repo. In addition to reducing the number of places to find a sample, including it in the samples repo would have the added benefit of raising awareness of all the other Streams samples among Beam only users. We could add a separate downloadable zip for the Beam samples to the releases page to make it easier to get the samples without cloning the whole repo.

In any case, samples can be added to the catalog even if they're not part of the repo, there are already about 10 or so of those in the catalog.

[natashadsilva]

If we do decide to go with a separate repo, I would call it streamsx.beam and have the samples as the only top-level folder to start with. streamsx.beamsamples might send people off in search of a project for the beam code, which isn’t open source.

[mrshenli]

@natashadsilva Hi, after a discussion in the Beam dev team, we prefer to have a separate streamsx.beam repo to host samples for now (and potentially the entire beam runner in the future). Are there any concerns with this approach? And do you know how we can make sure that those samples can be picked up by http://ibmstreams.github.io/samples/?

[natashadsilva]

Hi, I don't have any concerns. The samples catalog is updated manually you would need to send me the list of samples you would like to include, and I will add them.

[mrshenli]

@natashadsilva

Thank you! How do we create a new repo? It seems I do not have the access.

[schubon]

I just created https://github.com/IBMStreams/streamsx.beam. Let me know if you can work with this, or if you need some changes.

Regards Norbert

[mrshenli]

Hi Norbert @schulz2, thank you very much, It works perfectly!