IBMStreams / streamsx.topology

Develop streaming applications for IBM Streams in Python, Java & Scala.
http://ibmstreams.github.io/streamsx.topology
Apache License 2.0
29 stars 43 forks source link

What does the `populate` function do? #2540

Closed natashadsilva closed 4 years ago

natashadsilva commented 4 years ago

This function description just says "populate this topology" Source What does that mean? The explanation in the Composite class is confusing and needs some concrete examples to clarify what is meant.

markheger commented 4 years ago

What is missing?

ghost commented 4 years ago

populate creates the subgraph that is encapsulated by the composite. This would be the only thing that could be added to the documentation.

natashadsilva commented 4 years ago

Hi, I think the documentation could benefit from some simplification to take into account that someone reading it for the first time might not be is not familiar with SPL.

For example, the doc for Composite should start off with explaining at a basic level what it is used for. There should be a complete example of using a composite subclass within a topology. Even better would be to

There are other parts I found confusing:

Composites may use other composites during populate.

This makes it sound like the user needs to call the populate function? I don't think this is the case but it is worth stating explicitly.

Composites can control how the basic transformations are visually represented. By default any transformations within a composite are grouped visually. A composite may alter this using these attributes of the composite instance:

Grouped visually? What visual grouping is there when creating a Streams app in Python? Do you mean in the Streams graph at runtime? If so, this should be made specific.

I know it is a challenge to express SPL concepts while not using SPL terms directly but sometimes that is what is needed to aid understanding. This statement: populate creates the subgraph that is encapsulated by the composite is technically correct but "Subgraph", "encapsulated" and "composite" are not always familiar terms to the Python developer.

ghost commented 4 years ago

I improved the doc in the way that I make clear that users must implement populate in their composite classes to create their specific transformations.

Examples are already in the Python API documentation. The streamsx.documentation should link to streamsx.mqtt, streamsx.jdbc, streamsx.kafka, streamsx.objectstorage, and/or other Python packages, which contain implementations of Source, Map, and ForEach composites.