Improve the orchestrate ability of components

[yanghua]

Some survey research:

• https://github.com/ansible-community/ansible-bender
• https://github.com/ansible/ansible-container (DEPRECATED)
• https://github.com/openstack/kolla
• https://github.com/openstack/kolla-ansible

[yanghua]

IMO, the scalability has several levels:

• image-level: like this: https://github.com/openstack/kolla/blob/76d78b50c0208795c4bc93b932e7fb0e79a036fc/docker/grafana/Dockerfile.j2 which let user choose different versions;
• compose-level: multiple images orchestration also uses the ability of the Jinja template;
• container-level(runtime): use ansible container (Jinja + playbook) to do some initial works (lakehouse configuration, and so on) in the cluster;

@pan3793 WYDT?

[pan3793]

I would expect an image-level solution.

My original idea is very simple, use a template engine to render Dockerfile, files/, docker-compose.yaml, etc. that we write by hand currently.

[pan3793]

The idea is similar to helm, using the template engine to render the final yaml files which describe the resource deployed to K8s.

[yanghua]

I would expect an image-level solution.

My original idea is very simple, use a template engine to render Dockerfile, files/, docker-compose.yaml, etc. that we write by hand currently.

Ok, so your thought is static rendering. Let me think if there is any scenario that requires dynamic rendering.

[yanghua]

Let's discuss the potential split sub-tasks, my thought is:

• Introduce ansible and simply refactor Dockerfile via Jinja2 template to pass through the whole flow;
• Extract some existing hard code work into templates, which means deep refactoring;
• Through the above two steps, we can infer a unified mechanism for the scalability in the future;

WDYT? @pan3793