search

syndesisio / syndesis-project

Placeholder repository for project management
https://syndesis.io/
Apache License 2.0
6 stars 12 forks source link

As a citizen, leverage custom steps within Syndesis #152

Closed kcbabo closed 6 years ago

kcbabo commented 6 years ago

As a citizen integrator, I sometimes need to leverage custom steps that developers on my team have created in order to solve an integration problem. To do so, I need the ability to manage how custom steps are added to the platform through technical extensions and leverage one or more custom steps within integrations I create.

Developers package technical extensions as jar files, so I need the ability to import technical extension jar files into my Syndesis environment. The import process should validate the technical extension and allow me to review details of the extension (name, description, steps included) before importing. Since my company may develop more than one technical extension, I need the ability to list all technical extensions currently installed in my environment along with the ability to view details (similar to import view) of each technical extension. It should be possible to update a technical extension to incorporate bug fixes or additional features in the steps it contains. It should also be possible to delete a technical extension that is no longer used - I expect the Syndesis environment to warn/prevent me from deleting a technical extension that is currently in use.

Any steps included in technical extensions in my environment should be displayed as normal steps within the integration editor, including name and description. My expectation is that invoking a custom step in an integration behaves the same way as a step provided OOTB. If a custom step requires a specific data type and/or changes the data type, then I expect that the integration editor will be aware of this information and update type-aware steps (e.g. filter, data mapper) accordingly.

lburgazzoli commented 6 years ago

The story says:

"The import process should validate the technical extension and allow me to review details of the extension (name, description, steps included) before importing"

So if I properly understood the technical workflow would be:

Correct ?

kcbabo commented 6 years ago

Yes, that's correct. From a UX standpoint, this generally takes the form of a review/confirm page, IINM. We don't have to call it "approve" necessarily.

dongniwang commented 6 years ago

So the basic user flow would be: Upload tech extension file -> review info/data extracted from the uploaded file -> confirm import of the tech extension file

It should be possible to update a technical extension to incorporate bug fixes or additional features in the steps it contains.

@kcbabo What do you mean by the ability to update a tech extension? I thought we are not allowing editing of tech extension after it's been uploaded. Is update = delete existing one and upload a new one?

kcbabo commented 6 years ago

No, update would be to replace the existing one. I don't remember reaching agreement that update was out of scope. @rhuss @chirino - can you comment?

gashcrumb commented 6 years ago

Yeah, my recollection is you can update a tech extension and it replaces the existing upload however integrations at runtime run with a copy of the extension, so the user would have to redeploy an integration that was running with that extension to get the update deployed.

gashcrumb commented 6 years ago

And hopefully we'll be able to eventually flag an integration as needing update etc. so the user knows an action is necessary but that's part of the upgrade story :-)

kcbabo commented 6 years ago

Thanks, Stan. That matches my memory of the discussion.

dongniwang commented 6 years ago

No, update would be to replace the existing one.

How is this different from deleting the existing one and uploading one? Would updating a tech extension keep the same tech extension name and description? Or the backend understand that this is a newer version of the existing tech extension?

Yeah, my recollection is you can update a tech extension and it replaces the existing upload howeverintegrations at runtime run with a copy of the extension, so the user would have to redeploy an integration that was running with that extension to get the update deployed.

So let say if you have a running integration using tech extension A, the user action of replacing tech extension A to tech extension A2 won't break the running integration?

kcbabo commented 6 years ago

How is this different from deleting the existing one and uploading one?

From a UX perspective, I think the difference is significance. If a user wants to update a tech extension, forcing two steps (delete, then upload new) seems a bit severe.

Would updating a tech extension keep the same tech extension name and description?

Question about name is a good one as I don't know if we are treating the name as a unique id. If so, then I would say the name would be the same. Description seems like it could change.

Or the backend understand that this is a newer version of the existing tech extension?

Not sure if the backend will support versioning of tech exchanges - best for engineering to answer this one.

lburgazzoli commented 6 years ago

Extension metadata have:

What matters for the backend is the id and version so both name and description can be changed, this is left to the expert developer. If needed we can produce a warning as part of the extension validation step if we detect a change of the name/description. About version, as we have such information, we can detect that the extension is an update of an existing extension so we could also generate some information like listing the integrations that are impacted by such update as part of the extension validation.

The question now are:

nicolaferraro commented 6 years ago

Update is not the same as delete+create also because there should be restrictions on what can be deleted. In fact Syndesis should "warn/prevent me from deleting a technical extension that is currently in use".

About "warn/prevent", I think a safe restriction for us would be to "prevent" deleting integrations that are currently used. Otherwise we should define what would happen if a user tries to edit a integration that is using a deleted extension.

On update, we can put the additional restriction that new extensions should be "compatible" with the extension they replace w.r.t. what is currently used in integrations. E.g. if the new extension is missing a step that is being used by a integration, the new extension cannot replace the old extension. But we can also ignore the problem on update and force the user to fix the integration when he'll try to upgrade it.

Will this be part of another story or should we discuss about problems like this here?

kcbabo commented 6 years ago

There's a discussion here around backend internals that I won't offer an opinion on. If there's a question from a product capability standpoint, then I'm happy to answer. Generally speaking, more flexibility and functionality is better for features, but I realize there are tradeoffs in terms of cost. Having multiple versions, ability to revert, track usage, etc. are all great. What's the cost though?

If we need a quick phone call to work through these issues, I'm happy to set something up.

chirino commented 6 years ago

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its recipients. This is a temporary error. The following address(es) deferred:

chirino@gmail.com Domain hiramchirino.com has exceeded the max emails per hour (161/150 (107%)) allowed. Message will be reattempted later

------- This is a copy of the message, including all the headers. ------ Received: from o8.sgmail.github.com ([167.89.101.199]:32090) by host313.hostmonster.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89_1) (envelope-from bounces+848413-18a6-hiram=hiramchirino.com@sgmail.github.com) id 1f0Nme-0013ZM-Vk for hiram@hiramchirino.com; Mon, 26 Mar 2018 02:47:17 -0600 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=github.com; h=from:reply-to:to:cc:in-reply-to:references:subject:mime-version:content-type:content-transfer-encoding:list-id:list-archive:list-post:list-unsubscribe; s=s20150108; bh=c/WcOysujeaToHDXpeNvKSKN6EE=; b=ImhIyknCQA1yGZNe 0rM/DrWfQWM56jM+fYh7v3/BbXy/2+Of1+DrTfCoLeg5ZARq97pg1fgcrkh+/T1i nPTGODr4CcVoxvdEvuaWUewTr2wCHZuxnt54GW/lE1zah3ZJvZbnGfprvjPg0trd pY8Qa75Y20I2oVqfgaLAXGQC6To= Received: by filter0961p1mdw1.sendgrid.net with SMTP id filter0961p1mdw1-9203-5AB8B389-4 2018-03-26 08:47:05.2060751 +0000 UTC Received: from smtp.github.com (out-1.smtp.github.com [192.30.252.192]) by ismtpd0035p1mdw1.sendgrid.net (SG) with ESMTP id tsF6uch0QXSoj2R3ajNktA for hiram@hiramchirino.com; Mon, 26 Mar 2018 08:47:05.011 +0000 (UTC) Date: Mon, 26 Mar 2018 08:47:05 +0000 (UTC) From: Zoran Regvart notifications@github.com Reply-To: syndesisio/syndesis-project reply@reply.github.com To: syndesisio/syndesis-project syndesis-project@noreply.github.com Cc: Hiram Chirino hiram@hiramchirino.com, Mention mention@noreply.github.com Message-ID: syndesisio/syndesis-project/issue/152/issue_event/1540317105@github.com In-Reply-To: syndesisio/syndesis-project/issues/152@github.com References: syndesisio/syndesis-project/issues/152@github.com Subject: Re: [syndesisio/syndesis-project] As a citizen, leverage custom steps within Syndesis (#152) Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="--==_mimepart_5ab8b388ae04f_2b6a2ae227d32ec420276d"; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: list X-GitHub-Sender: zregvart X-GitHub-Recipient: chirino X-GitHub-Reason: mention List-ID: syndesisio/syndesis-project List-Archive: https://github.com/syndesisio/syndesis-project List-Post: mailto:reply@reply.github.com List-Unsubscribe: mailto:unsub+000193578f00778347e6b0e833d6a01bf9a72490c8c188dc92cf0000000116d0758892a169ce0fffadb2@reply.github.com, https://github.com/notifications/unsubscribe/AAGTV-yZKipZikrUfHqa24HscRqnIzArks5tiKsIgaJpZM4QGEwj X-Auto-Response-Suppress: All X-GitHub-Recipient-Address: hiram@hiramchirino.com X-SG-EID: r3yY3NeKU5c391Z9JqIJsAQ+rIWE1mJvBRhY4sfmQMtIBe+Ap9MZOPWKaxIb4KOcC8CrgTAme8QcC7 xSPTbDSLEsa46oqBinDorJEjwpk3TUUl3LNFL//UgYKHrV7Y498IkJ2niKNeD5U13fnG7iYCj7Dbe6 LAuj/wLasq2GJ/5RCj37MCbXNpEZylJTKxQV5MoeiJaIcR/piz4KPyFyZGUUwMeIQKpvdFDV8K0kLa A=

----==_mimepart_5ab8b388ae04f_2b6a2ae227d32ec420276d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit

Closed #152.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/syndesisio/syndesis-project/issues/152#event-1540317105 ----==_mimepart_5ab8b388ae04f_2b6a2ae227d32ec420276d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Closed #152.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

----==_mimepart_5ab8b388ae04f_2b6a2ae227d32ec420276d--