Create a task to provide the capability to query db instances

[morarucostel]

Is your request related to a problem? Please describe. The business needs an easy way to access data from a mongodb instance in order to perform additional analysis.

Describe the solution you'd like A workflow task capable of connecting to a mongodb instance and run a query. The output of the query must be accessible as a file at the end of the task execution.

Describe the benefits or justification for this request Increases the capability of the platform and opens the door to access the data without any prior transformation, allowing the transformation to be done post retrieval, in the flow.

Additional context NA

[tlawrie]

Hey @morarucostel thanks for the issue. Its not currently tagged to a release but do see it included in a loader change

https://github.com/boomerang-io/flow.loader/pull/67/files#diff-c85a304c1ca5ffe621b41e6cde3f9667226cc680991cc29115518fe5edb393e5

Should this have been reviewed before being included in the same loader change as the HTTP Task Template changes?

[tlawrie]

cc: @morarucostel @amhudson

[morarucostel]

Hey @tlawrie , yes, it should have been reviewed before being included. That is my fault, I jumped the gun there. We can either revert it or review it part of this issue. The task is using the mongosh image from mongodb and the UI is configured like this:

[tlawrie]

Hi @morarucostel this is a good addition. Its great new tasks are being generated. Not a problem, we should probably devise a process for new tasks and a +1 or +2. We havent really ever had a process for it.

W.r.t to this task, few comments

1. If it's using an external image, we might need to document that. Im thinking we either add a list of dependent images in the architecture documentation OR add an extra column to the tasks. I think it might be better in the architecture, to add images that we depend on for tasks.
2. How does this task work for TLS required MongoDBs?
3. Some more helper text or placeholders might be good. In the query it could have an

   e.g. example query

[morarucostel]

@tlawrie points 2 and 3 are solved (attached as images below) Regarding point 1, documenting the dependent images, do you think we can stick them in 3.6.1 release docs?

[morarucostel]

@tlawrie also exposed, as configurable, the auth method so now the task can be used to connect to both internal in k8s hosted mongodb instance as well as external Azure hosted instance. The current configuration is:

[morarucostel]

@tlawrie also updated the Architecture - Application - Components documentation with the dependency on the MongoDB shell docker image. It made sense to add the information under the already existing external dependency column of the TaskWorker.

[morarucostel]

Is it OK to tag it against 3.6.1 version? The task template and documentation are already done.