add training portal environment objects

[djschny]

Is your feature request related to a problem? Please describe.

There are objects that can be defined at the session level for a workshop, and there are also objects that can be defined at the workshop environment level that can be referenced/shared across workshop sessions. It would be nice to have objects that could be defined at the training portal level and referenced/shared across all workshops associated with the training portal.

An example of this would be things like the following:

• credentials/IDs that is shared across workshops
• series of workshops that share the same backend system

Describe the solution you'd like

I believe this is explained above.

Describe alternatives you've considered

Manually scripting/managing the lifecycle outside of the training portal applying and deleting.

Additional information

No response

[GrahamDumpleton]

This idea has been looked at before and although it would seem to complement what can be done with workshop environments and workshop session, there was concern it would be abused to do things it shouldn't really and which are better off left to other tools which are more suitable. For example, it might look appealing to use this to install a cluster wide Kubernetes operator, but in practice that could cause a lot of problems when training portal instances are deleted and re-created because of the CRD definitions.

In the main it was thus felt for cluster wide deployments (eg., operators, or other services), or even just configuration which is used by many workshops, potentially across different training portals, that using kapp-controller from Carvel separately is still a better solution.

This doesn't mean we couldn't still add the feature, but it would be recommended that one really limit quite a lot what you should use it for, which doesn't make it as compelling. So would need to see a good demonstration of a use case which really justifies adding it, over using kapp-controller. As such, can you expand on your examples and how having the lifecycle linked to the training portal makes it more useful.

cc @jorgemoralespou

[jorgemoralespou]

Agreed with @GrahamDumpleton. A more detailed example of why this is needed could help us understand better whether this is an option. In the past, for some common use cases we have extended the spec, e.g. registry so that what's added is controlled by the platform. I agree that allowing random objects to be created and bound to the training portal is a recipe for disaster, and while there will be advanced users that know what they are doing, there might be others not so advanced that use this mechanism as convenience installation mechanism and potential source for disaster. All this can be currently done by the platform/infrastructure operator as a side task.

But having more detailed request might help us understand better.

[djschny]

As such, can you expand on your examples and how having the lifecycle linked to the training portal makes it more useful.

A more detailed example of why this is needed could help us understand better whether this is an option.

One concrete example would be a course made up of a series of workshops (ie. 5 or 10) where there is a workshop for each. Each lab/workshop makes use of a readonly database filled with data. If done at the workshop environment level, the same thing would be started up and running 5 or 10 times (equal to the amount of workshops). By having one at the training portal level it could be shared across all the workshops and save resources in the cluster. It's not appropriate or a platform/infrastructure operator to install that at the cluster level because the configuration and setup of that is specific to the educator who creates the workshops.

The same above concept would apply to requesting resources externally as well (Shepherd leases, AWS/cloud resources, etc) that would be used and shared across all the workshops.

[jorgemoralespou]

@djschny what would be the lifecycle of these resources in case of a trainingportal update that adds an update to these resources? What should happen if that readonly database is changed while there are tenths (or more) of ongoing workshops? Users could get different results when the new database has been rolled out. Even though on one side I do agree this is part of the educator to me it feels like shared infrastructure that needs to be rolled out at a known time or at least understand the implications for running sessions. It's similar to the scenario of a workshop that required knative. Should knative be added as these resources by the trainingportal, or should be part of the documentation/requirements for the cluster where this trainingportal will live. It's a thin line that's hard to define. Also, what happens if multiple trainingportal defines the same set of resources to be added globally to the cluster? Should we limit these resources to namespaced types of resources that they are added to the trainingportal NS so that they can be consumed only from the trainingportal to which the environment/session belong? Food for thought!

[djschny]

what would be the lifecycle of these resources in case of a trainingportal update that adds an update to these resources?

It would follow that of the training portal being update.

What should happen if that readonly database is changed while there are tenths (or more) of ongoing workshops?

It depends upon what is changing. If it is minimal, the content author may choose to just let it naturally happen, if it's something larger or more major, then it could be having it go out as a new training portal deployment and then after time removing the old training portal.

Users could get different results when the new database has been rolled out.

That's data management within the database and is the concern of the content author. They may mitigate with making different database names in the database with the different data. This new database name would be used in the updated instructions. TBH stuff like this is a content authoring item that needs to be addressed even without using a feature like this.

It's similar to the scenario of a workshop that required knative. Should knative be added as these resources by the trainingportal, or should be part of the documentation/requirements for the cluster where this trainingportal will live.

That's a completely different topic, as that's installing clusterwide kinds of things. That's not in scope for a use case like this.

Also, what happens if multiple trainingportal defines the same set of resources to be added globally to the cluster?

That would not be in scope for this. In these situations the resources deployed would either go into a namespaces that is unique to the portal instance or be named similarly.

Should we limit these resources to namespaced types of resources that they are added to the trainingportal NS so that they can be consumed only from the trainingportal to which the environment/session belong?

No, that would be up to the educator who configures those resources and not a concern of educates (unless I'm missing something).

Food for thought!

Thanks, happy to discuss these and more.

I mean I can accomplish this by building more intelligence into deployment pipelines and workflow automation. I was going to start down that route, but thought I would file a request here since it would help folks who might not have the tools and resources at their disposal that myself and my organization has.

[GrahamDumpleton]

When you say:

By having one at the training portal level it could be shared across all the workshops and save resources in the cluster. It's not appropriate or a platform/infrastructure operator to install that at the cluster level because the configuration and setup of that is specific to the educator who creates the workshops.

The problem here is that the educator who creates the workshop only really has control over the workshop definition and not the training portal, which currently is a separate thing that an administrator would usually only manage.

As described in:

• https://github.com/vmware-tanzu-labs/educates-training-platform/blob/develop/developer-docs/project-roadmap.md#simplified-course-deployment

we have been looking at introducing a separate concept of a course which would consist of multiple workshops.

This resource would list all the related workshops, but also provide parameters which would be use to fill out the training portal for the course when created, or perhaps one might even allow one to inject the set of workshops into an existing training portal instance.

So if we went down the path of having this ability to create resources common to a set of workshops, it may make more sense for it to be part of the course resource and not the training portal.

In other words, when deploying the course, it defines the common resources required by the set of workshops and then also triggers the creation of the training portal as well.

If one edited the course and added or removed workshops, they would in turn be added or removed from the associated training portal.

So the training portal becomes an internal implementation detail which the course creator still doesn't need to worry about directly.

In order though to still allow an administrator to customise that training portal instance, the course definition should perhaps be similar to a workshop definition, in the sense that loading it into a cluster wouldn't actually do anything. One would still use a separate resource (TrainingProgram, or some other appropriate name) at a higher level to initiate deployment of the actual training portal and set of workshops against it. This higher level resource might even allow one to aggregate multiple courses, or give control over spreading them across multiple training portals or clusters, with a higher level REST API which spans across multiple training portals on the same or separate clusters.

Modelling it this way thus still provides a good separation between stuff the course creator defines and then how it is deployed by an administrator.

[djschny]

The problem here is that the educator who creates the workshop only really has control over the workshop definition and not the training portal, which currently is a separate thing that an administrator would usually only manage.

FWIW - That's not an always true statement. In my organization and team the training portal is part of the content/workshop creator's control. We have it live in the same repo as the workshop content and configuration.

[GrahamDumpleton]

That is probably only because there is no better alternative right now when have set of workshops which need to some how relate, although in practice you didn't need to use the Educates TrainingPortal resource for that, and could have had your own meta data object which tracks the association. IOW, your own Course object. Adding a Course resource type into Educates would just formalise that abstraction.

[djschny]

FWIW - even if the Course object were to be created, we would still want the training portal configuration items to be controlled by the workshop creators. Unless I'm unclear on what fields would move out of the training portal definition.

[GrahamDumpleton]

The training portal wouldn't change. The course object would be an abstraction to separate how stuff gets deployed from the what is required. The course object would still be able to specify workshop durations, timeouts, or periods for extension etc, and so would act as a template for filling out the training portal, but the training program resource I mention which is then used to initiate deployment of the course, would also allow overrides for stuff related to workshops where administrator felt they needed to customise something because of their knowledge of the cluster setup, eg., capacities. So the training portal is created from information in the course object which course creator manages with overrides from the training program object which administrator manages.

[jorgemoralespou]

In my organization and team the training portal is part of the content/workshop creator's control.

This is not common, and also not what is encouraged by the educates team. There's obviously times where the same person will have both roles, but responsibility wise (which is the decision that affects design) they are separate tasks that should be achieved by different roles.

The idea is that workshops can be combined in different trainingportals, if needed, as workshop provides the content and the traininportal provides the access point and capacity management, which higlights the distinction of the roles involved.