Support HOT (Heat Orchestration Templates)

[tliron]

See the spec.

HOT can make use of the YAQL language. We will not support it in the first stage as part of this issue, as this would require rewriting that language parser in Go.

[abhishek-kumar0409]

Tal, it will be really cool to have this feature in puccini. I was looking for something like this in puccini. Tal, if you don't mind, can you please brief about the scope of this requirement?? Are you planning to develop a plugin to translate a tosca-template to heat template orchestration or is it something else ?

[tliron]

Progress happened today. The feature is remarkably easy, because HOT is a very trivial language compared to TOSCA. So a lot of code is easily reused.

The way it will work is:

1. puccini-tosca compile will be able to accept HOT transparently. It looks for the heat_template_version string if the tosca_definitions_version string and handles it accordingly. So, not too different from choosing the right version of TOSCA.
2. The result will thus be a Clout file that looks exactly the same as if you used Puccini's OpenStack profile for TOSCA. All the same tooling would work.
3. What does it mean in practice? For now, I'm experimenting with producing Ansible playbooks to deploy on OpenStack. They're still in PoC stage, but they do work. This means that Puccini allows you to bypass Heat while getting the same final result. That in itself in not super useful. But then of course you can completely customize those playbooks and do much more interesting integrations.
4. I also want to target Kubernetes. How would that work? KubeVirt would handle the VMs and Multus would allow for Neutron-like capabilities. So basically an existing Heat template would be deployabale on Kubernetes. Well, almost. :) There are still a few gaps. I've discussed some aspects of this in the ONAP-OPNFV meetup in France last month. I will go into some more detail in a talk I am giving in ONS North America in April. Here's a link if you can make it.

[codesnk]

This confuses me. If I want to create a service, do I start writing TOSCA or HOT or Helm? With Puccini, I thought the approach was to use TOSCA (as a language spec, not its different profiles) as the common way of designing services for different cloud platforms. May be ONAP is to blame for its use of TOSCA, HOT, BPMN etc. but can you give a context on why HOT is now going to be processed like TOSCA in Puccini? Equivalently will we see Puccini consuming K8S resources in future alongside TOSCA and HOT? Also not sure why bypassing Heat is targeted. May be my fuzzy knowledge on the OpenStack side of things and the ever increasing complexity of ONAP is defeating me here.

Ps: I will be waiting for a video recording of the talk at ONS.

[tliron]

If I want to create a service, do I start writing TOSCA or HOT or Helm?

I would strongly advise TOSCA, as it is the most forward-looking, open, and flexible.

HOT is being introduced to support legacy OpenStack applications.

As for Helm -- well, I don't personally have a lot of good things to say about it. In itself it's not much different then writing pure Kubernetes specs, except that it introduces text-based templating, which is entirely the wrong way to make YAML "flexible". Its Tiller component adds some lifecycle management, which I think is confusing. You are better off writing native Kubernetes operators. The bottom line is that I see Puccini plus its Kubernetes profile as a replacement for (future competitor of?) Helm.

Equivalently will we see Puccini consuming K8S resources in future alongside TOSCA and HOT?

Probably not. :) Support for HOT was added because it's a TOSCA-like language or dialect, in fact one that is much more widely used than TOSCA at the current time. Another TOSCA-like language that could be supported is Cloudify DSL. But that would depend on interest.

Also not sure why bypassing Heat is targeted.

It's not Heat that I hope to bypass: it's all of OpenStack. Imagine having your clouds running Kubernetes, either directly on bare metal (at the edge?) or on top of other cloud infrastructures, both private and public. The idea is to take an application (for example, a VNF) that has already been packaged via HOT for OpenStack and deploy it, as is, to Kubernetes. This is the most exciting use case.

But still, you could potentially deploy on OpenStack via Puccini + HOT. What is the advantage of getting rid of Heat? Well, by bringing in your own alternative orchestrator you might be able to do deeper integrations. For example, you might want to use a BPMN-based controller to integrate with business processes and create an open control loop. That's not something you can easily do with Heat. This is, perhaps, less exciting than the previous use case.

[codesnk]

I now understand the context for this HOT support. Thanks for the explanation. I don't think I will be using it though :)

The thing I like about puccini is that it enables you to start with pure TOSCA and generate the platform resources directly. This should be the way for all cloud platforms. The need for tiller like LCM does exist however. Wouldn't that be a great addition to Puccini :)

I have used puccini for TOSCA-to-K8S resources generation and it just works (with extensions of course). I also agree with you on Helm and the view on K8S operators but the legacy ETSI MANO and operator driven projects like ONAP is not allowing a cleaner path like that.

[tliron]

Thanks! LCM additions are necessary but beyond the scope of Puccini. I have another project, though, that might be the right place to add those. I will talk about it a bit in the ONS presentation.

[codesnk]

Great.. The LCM related project you mentioned, is it open source or cooking? Or we have to wait till after ONS?

On this HOT topic, I think its makes sense to place higher priority on supporting a converged Container+VM cloud platform where TOSCA language spec is the service definition starting point. One could think of the TOSCA profiles like programming libraries that a designer uses to design a service and then parse/compile through puccini to get the actual platform resources managed by dedicated controllers. Considering that projects like Kubevirt, Amazon firecracker, Kata etc., are bridging the Container, VM divide, it is probably a good time such a cloud platform (running on bare metal in my point of view) becomes the target for future developments in this area.

[tliron]

The project I have in mind is still just an initiative.

You are absolutely right about what we can do with Kubernetes on bare metal. See Intel's work in the Multi-Cloud project of ONAP: right now you can use ONAP to deploy a VNF to Kubernetes with Virtlet and Multus. And yes, of course I see TOSCA as being the right language to do all these different things while staying on the same tool chain. This is the whole point of Puccini.

As for HOT, you can really just think of it as a popular dialect of TOSCA. For the purpose of Puccini it is "low-hanging fruit" that is easy to implement and can help increase Puccini's reach.

[tliron]

Essentially working in Puccini 0.6.