Proposal for new toolkit streamsx.cloudant

[brandtol]

Cloudant is an IBM owned non-sql database, available in Bluemix and as on-premise version. It stores JSON documents (and possibly binary attachments) identified by a key and a revision number. (https://en.wikipedia.org/wiki/Cloudant).

During recent projects the need came up to have an interface from IBM Streams to Cloudants "Changes" API. This API allows users to connect and receive all database changes as they happen, in a continuous feed. One use case for this is to synchronize (and possibly transform) data from Cloudant to another database. (https://console.bluemix.net/docs/services/Cloudant/api/database.html#get-changes)

I propose to create a new toolkit named streamsx.cloudant, that initially contains three operators:

• ChangesSource connects to the changes API and ingests changed document IDs into Streams (the operator shall be usable as start of a consistent region)
• DocumentSource fetches a document from the database based on its ID. The document is send downstream as JSON string.
• DocumentSink stores a tuple (or JSON string) as document in the database

The toolkit will use the official Cloudant Java client library (https://github.com/cloudant/java-cloudant).

[ddebrunner]

+1 with usual caveat that operators are named on what they are doing, not just Source and Sink, e.g. DocumentFetch, DocumentStore.

[mikespicer]

+1

[leongor]

+1

чт, 19 окт. 2017 г. в 0:03, Mike Spicer notifications@github.com:

+1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/IBMStreams/administration/issues/127#issuecomment-337727238, or mute the thread https://github.com/notifications/unsubscribe-auth/AGvlA77MWu9AxxOJgfRZOVFzCWqDgZLtks5stme4gaJpZM4P-DBI .

[chanskw]

+1

Suggestion to name the operators as follows: ChangeSource -> ScanDocument (similar to directory scan) DocumentSource -> FetchDocument DocumentSink -> StoreDocument

[chanskw]

Can you please let me know who should become a committer to this project?

[chanskw]

Created repo, please let me know if you need to add more committers.