Open simardst opened 4 years ago
@simardst This is a very interesting scenario and we would be very happy to solve it in OAM.
Just to confirm, is the workload controller your component rely on is a customized k8s controller? So in that controller you have the logic to do different things based what kind of trait it is bound with, correct?
No, not really.
In fact, we are not using OAM yet. We are deploying containerized applications that run on a edge device using a in-house controller (not k8s). As for OAM, the applications are composed and configured by the application operator.
We are investigating how OAM could allows us to simplify and clarify the development of reusable containers, the composition of them into applications, and the deployment and configuration/parametrization of applications.
Therefore, we are trying to define some of our existing components and applications using the OAM constructs (ContainerizedWorkloads, Scopes, Traits, ExtendedWorkloads, ...)
Using k8s to deploy our apps on our edge devices is an option, but short-term we are planning to implement a limited set of OAM concepts using custom controllers built from our existing stack.
short-term we are planning to implement a limited set of OAM concepts using custom controllers built from our existing stack.
@simardst That would be the reasonable approach IMO, k8s controllers are mostly too heavy for small devices.
So in that sense, the most straightforward approach is OAM runtime (let's say OAM IoT runtime) patches the index of AppConfig to the Component's metadata.annotation
, so the Component impl could fetch the AppConfig and check corresponding traits list to do the job.
In OAM k8s runtime, this relationship is tracked by metadata.ownerReference
.
Speaking of the OAM IoT runtime, it would be really helpful if you could share its design. We are eager to update the OAM spec to work better with non-k8s IoT scenarios (for example, define a bunch of standard annotations to track relationship of objects).
If I understand, you are suggesting that we should define an ExtendedWorkload for our containers component (very similar to the ContainerizedWorkload), and that the associated controller should have custom logic to inspect the parent ApplicationConfiguration to detect that a VolumeTrait is applied.
In that case, it should inject something into the component, ie. the container spec (ex. a environment variable) to inform it what class/type of storage is bound to it?
@simardst You don't need to define ExtendedWorkload, I am referring to leveraging the metadata.annotations, this is the place where system level informations are patched to.
For example:
apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
name: frontend
annotations:
appconfig.oam.dev/name: my-edge-app # <--- this annotation is auto patched to the Component object by OAM IoT Runtime.
spec:
workload:
...
Ok, but we still need some special logic into the controller of the ContainerizedWorkload to inject something into the component, ie. the container spec (ex. a environment variable) to inform it what class/type of storage is bound to it?
@simardst So it seems it's actually your code relies on such environment/information to react differently to different storage types, not the Component controller, correct?
In that case the app operator can just use parameterValues
to set the ENV of Components, for example:
apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
name: frontend
spec:
workload:
apiVersion: core.oam.dev/v1alpha2
kind: ContainerizedWorkload
spec:
containers:
- name: my-cool-workload
image: example/very-cool-workload:0.1.2@sha256:verytrustworthyhash
env:
- name: STORAGE_TYPE
parameters:
- name: storageType
required: true
fieldPaths:
- "spec.containers[0].env[0].value"
---
apiVersion: core.oam.dev/v1alpha2
kind: ApplicationConfiguration
metadata:
name: my-app-deployment
spec:
components:
- componentName: my-edge-container
parameterValues:
- name: STORAGE_TYPE
value: local
That being said, better understanding on how you implement OAM on IoT would really helpful since I am not very sure how your "controllers" works and the bigger picture (e.g. how trait and workload are implemented etc).
That's exactly the solution we came to at first. The problem we see is that the application operator has to know that, if he sets the STORAGE_TYPE parameter to 'remote', he also has to specify a compatible StorageClass when applying the VolumeMount trait.
For instance, he has to know that he cannot set STORAGE_TYPE = 'local' and apply a VolumeMount that binds the component to a NFS volume.
We are expecting a way to reduce or eliminate that kind of errors. Maybe a way to add constraints or validity requirements on the traits that can be applied to a specific component.
@simardst Got it. Thank you for making the issue more clear!
The most straightforward way is using a feature named data input/ouput in OAM, for example:
apiVersion: core.oam.dev/v1alpha2
kind: Component
metadata:
name: frontend
spec:
workload:
apiVersion: core.oam.dev/v1alpha2
kind: ContainerizedWorkload
spec:
containers:
- name: my-cool-workload
image: 'example/very-cool-workload:0.1.2@sha256:verytrustworthyhash'
env:
- name: STORAGE_TYPE
---
apiVersion: core.oam.dev/v1alpha2
kind: ApplicationConfiguration
metadata:
name: my-app-deployment
spec:
components:
- componentName: my-edge-container
dataInputs:
- valueFrom:
dataOutputName: vol-type
toFieldPaths:
- 'spec.containers[0].env[0].value'
traits:
- trait:
apiVersion: core.oam.dev/v1alpha2
kind: VolumeMounter
spec:
volumeName: myvol
storageClass: local
dataOutputs:
- name: vol-type
fieldpath: 'spec.storageClass'
As you may noticed we already implemented this in OAM Kubernetes Runtime and now working on adding it as part of the OAM spec (as experimental fields).
But since you are not using k8s for now, implementing this feature in in-house controller could be complicated. So in short term, I would suggest do some hack in your controller:
WDYT?
@simardst Another option is we can refactor out the core functionalities of oam-kubernetes-runtime as lib so you can use it to build non-k8s runtime directly.
We have a component that needs to adapt its behavior in function of the type of storage bound by the application operator when deploying the application. The component needs to know if the storage local ou remote (nfs).
As we understand, the volume binding would be accomplished by the application operator using a trait (VolumeMount) in the ApplicationConfiguration.
Another option we are considering is to create 2 versions of the application (one for each type of volume).
Finally, what would be the recommended way to handle this case?