search

oasis-open / tosca-community-contributions

OASIS TC Open Repository: Manages TOSCA profiles, tests, and templates that are maintained by the TOSCA community. They are intended to be used as examples to help developers get started with TOSCA and to test compliance of TOSCA implementations with the standard.
https://github.com/oasis-open/tosca-community-contributions
Apache License 2.0
37 stars 25 forks source link

Can we get rid of (imperative) workflows? Do we need to replace them with topology-wide event definitions? #74

Open aszs opened 2 years ago

philippemerle commented 2 years ago

Could "topology-wide event definitions" be more elaborated?

koppor commented 1 year ago

The event definitions are discussed at https://github.com/oasis-open/tosca-community-contributions/discussions/153.

Side comment: For workflows, I would always point to BPMN (or BPEL ^^) and not re-invent any workflow language. Usage of these technologies has been perfectly shown by the OpenTOSCA Container. Although the AI community comes up with other approaches such as Apache Airflow or Flyte.

tliron commented 1 year ago

@koppor Thanks for linking the discussion, and I tend to agree with you that we shouldn't have to reinvent wheels. Whether it's a BPMN workflow (we would need to define BPMN integration points with TOSCA, though, which I assume OpenTOSCA Container does) or an Ansible playbook (we would need to define TOSCA graph traversal tasks, which I've done in Puccini), these external files can be included as artifacts in CSAR.

Unfortunately for our opinion, we have received requirements from certain consumers of TOSCA that generic workflow grammar is needed for portability (in use cases that require portability, which I don't think covers all use cases). For example, if we standardize on BPMN we are saying that you need a BPMN workflow engine (e.g. Camunda). I don't think we want to require specific pieces of software for TOSCA, generally speaking (though specific domain profiles might). In order to satisfy these specific TOSCA users a basic workflow grammar is probably necessary.

koppor commented 1 year ago

Short answer: Yes, opentosca defined it. Kind of straight forward. "Just" interacting with the offered management Interfaces. -- IMHO, there could be TOSCA BPMN Profile with a subset of BPMN supported. There even is a Python-based engine: https://www.spiffworkflow.org/ -- There also is a (proprietary) C++-based Implementation. No need for a strong requirement on Camunda. This is what "portability" is for 🤣🤣. -- Seriously, BPMM is an OMG standard (like UML). Thus, it is certainly more wide-spread than a custom made workflow language. One might think that workflows are easy. Having seen development of standards and engines more than two decades now, I can assure, it is not. -- Finally, there is a workflow endinge in less than 1000 LOC: https://cpee.org/ (did not invesrigate thoroughly)

I need to think more about the graph traversal requirement. First thought: Isn't that a thing happening before? There needs to be a generator mapping from the topology (declarative) to the BPMM workflow (imperative).

tliron commented 1 year ago

See the discussion you linked, #153, for my most recent proposal from 2 days ago for an improved workflow definition grammar in TOSCA.

As to whether BPMN is the format we would want to rally around in TOSCA, ... I think it would be an uphill battle. We generally don't want to refer to too many other standards from TOSCA. The most we do is the YAML 1.2 spec, some IEEE specs for data types, and PCRE for the regex engine. But being opinionated about BPMN is not something I think most TOSCA users would support. BPMN (BPMN2 rather) has its own history with a lot of baggage. It's often used in higher levels of orchestration, such as OSS/BSS in telco. After all the "B" in BPMN stands for "business", not cloud. That's not to say, as you say, that there can't be a community-contributed BPMN profile. I think that could make a lot of sense for some users who do rely on BPMN already, or who want tighter integration with OSS/BSS.

Also see my small PoC to show how BPMN could be generated from TOSCA workflows (TOSCA 1.3 grammar), so that's also an option. Generally the point is that TOSCA-defined workflows, because they really are so minimal, could be consumed by any workflow engine. Or one can represent BPMN processes as TOSCA nodes and relationships. Not to say that any of this is a great idea necessarily, I think you would always be able to achieve better results writing workflows "natively". (Then again I'm opposed to workflows generally, I think they are terrible constructs that have no place in the cloud...)

(Side note: I'm not sure an enterprise would want to deploy thousands of sites using a 1000 LOC workflow engine...)