How to define function authorization?

[cdavernas]

How to define function authentication?

First of all, big up to all of you, the specification is just awesome.

Second, and even though timing is not perfect (as the spec is not even in RC), I developped my own .NET 5.0 workflow engine for the Serverless Workflow spec, in the context of a cloud native solution for one big member of the CNCF. So far so good, I can already define and run very complex workflows, but I recently ran into a big roadblock, conceptually speaking.

As a matter of fact, one of my workflows must invoke REST functions on a out-of-cluster, remote server, which exposes its OpenAPI document on a (poorly) secured endpoint.

I understand that, as opposed to what's explained here, Serverless Workflow does not aim to define/expose non-workflow specific info:

This approach, however, often runs into issues such as lack of portability, limited capabilities, as well as forcing non-workflow-specific information, such as service authentication, to be added inside the workflow language.

However, as Serverless Workflow specification relies on OpenAPI's for describing functions (which IMHO is a simple, elegant and brilliant idea btw), and that the latter does expose ways to define authorization, is there a way to leverage that? If not, what would you recommend doing for connecting to secured OAS endpoints and to execute secured functions?

---

Here are some ideas that came to my mind will writting this issue:

1. Declare the auth protocol and values to use when executing a function:

functions:
    - name: ListPets
      operation:  #OperationDefinition object or string such as 'https://securedendpoint/swagger.json#listPets'
         id: listPets
         definition: https://securedendpoint/swagger.json
         authorization:
            scheme: basic | bearer
            bearer: '${ .token }'

2. Declare the auth protocol and values to use when connecting to a secured OAS document

3. Declare and object used to reference external resource files, such as secrets

# workflow.yaml
id: MyWorkflow
version: 1.0
name : MyWorkflow
resources:
  - name: SECRETS
    file: mysecrets.json
functions:
    - name: ListPets
      operation:  #OperationDefinition object or string such as 'https://securedendpoint/swagger.json#listPets'
         id: listPets
         definition: https://securedendpoint/swagger.json
         authorization:
            scheme: basic
            username: '$SECRETS.username'
            password: '$SECRETS.password'
...

# mysecrets.yaml
{
   "username": "foo@bar.com",
   "password": "foobar"
}

The above example would allow securing sensitive data in external resources, and use some convention to use to replace variables defined in workflow.

[tsurdilo]

@Neuroglia First of all thank you for the nice words! Would you be willing to share your implementation with the community so we could promote it on our website (serverlessworkflow.io). We are working on adding an "Integrations" section there, so if you would like to we could add your project logo and some text of your choice and a link back to your project.

Regarding the auth types to support, maybe we can start with "basic" and then work from there to add api keys and oauth? WDYT?

Regarding secrets, I think we should maybe not allow users to define their own secrets.json/yaml that include plain text, but add ability to define domain-specific secrets from the runtime environmental variables

Once defined, then we could use your second example to access them in workflow expressions.

Do you think this is something you would like to help us define?

[cdavernas]

@tsurdilo Thank you for your prompt answer!!!

So, I'd be more than happy to share my implementation and to be able to receive feedbacks and possibly collaborate to it. We are working with my client on ways to open-source parts of the solution, but as you probably know, there is a lot of compliance involved and those things usually don't move very fast. I'll keep you guys posted on that subject, and will hopefully be able to release it soon enough.

In the meanwhile, would you be interested in a .NET 5.0 SDK project? If so, could you create a 'sdk-net' repo to which I could PR my sources? I could probably do it by the end of the week.

Concerning auth, we definitly could start with the 'basic' scheme, though IMHO 'bearer' is likely to be the one with the highest potential and value. In the end, I suggest to support following schemes:

• basic
• bearer
• openid connect (client_credentials and password grants only, for obious reasons)

Concerning secrets, I am not sure I understand what your are proposing with:

define domain-specific secrets from the runtime environmental variables Are you thinking of the workflow engine's environment variables? If so, wouldn't it imply restarting its pod every time new variables are added? As far as I'm aware of, in K8s, for example, you can only set environment variables for a specific pod at create/apply time. It would therefore mean that every time a new workflow definition with domain-specific secrets requirements is deployed, a pod restart might be required, no?

When you are asking about me helping/contributing, is it about that last part or about auth? I'd be really happy to help in both cases anyways, the project is just too good, I've got great faith in it!

[tsurdilo]

@Neuroglia 1) sdk-net: https://github.com/serverlessworkflow/sdk-net I will add you as maintainer of this repository. If you decide to contribute your sources (which would be really cool) I would like to ask you to please also maintain this repository. You will need to add the projects license header to your sources also. Here are some examples of that: https://github.com/serverlessworkflow/sdk-java/blob/main/api/src/main/java/io/serverlessworkflow/api/workflow/BaseWorkflow.java https://github.com/serverlessworkflow/sdk-go/blob/main/parser/parser.go

2) Regarding contributing, definitely not limiting you to anything specific, you are free and more than welcome to contribute to any / all parts of this project. The idea was just to have your input and help on the specific request in this PR if possible.

We discuss PRs and requests in meetings as well so everyone can give their opinions. If you can please join the meets when you have time. You can find the zoom link here: https://github.com/serverlessworkflow/specification#Meetings Feel free to also join our slack channel. You can find the link for that here: https://github.com/serverlessworkflow/specification#community

3) Regarding the secrets, I think we should find a way for users not to have to hard-code text-based credentials. Env vars seems to be predominately used but also volumes. I'd like to get everyone involved in this so we can try to find the best solution for this if possible.

[tsurdilo]

@jplane If I recall correctly, you are also working with .NET? Once @Neuroglia adds his code to https://github.com/serverlessworkflow/sdk-net it would be great to get your inputs on it as well. Thanks!

[ricardozanini]

@Neuroglia, first of all, many thanks for your contribution and for bringing this topic for discussion! (and also for your SDK to-be contribution).

If not, what would you recommend doing for connecting to secured OAS endpoints and to execute secured functions?

I believe the runtime implementation must handle this kind of feature, using the information provided by the given OpenAPI.

Since the OpenAPI spec already has this information, replicating it here is a no-go, IMHO.

To illustrate, you can see this example on Kogito: https://github.com/kiegroup/kogito-examples/tree/master/serverless-workflow-temperature-conversion

If the OpenAPI definition file has the security description, a user can easily provide the credentials using system properties, for example:

org.kogito.openapi.client.multiplication.base_path=${ENDPOINT}
org.kogito.openapi.client.multiplication.user=${USER}
org.kogito.openapi.client.multiplication.password=${PASSWORD}

Whereas multiplication is the IDs for the given OpenApi def. Same thing for OIDC. You can provide your keys there.

Kubernetes can handle this kind of property by mounting a configMap in the target pod with this info. A user can build a CI/CD pipeline to map this information and create the CM. Or you can use the same properties file and inject the USER and PASSWORD from an environment variable mounted through a secret.

Long story short, the security info is already provided by OpenAPI definition. I don't think we should replicate this info here.

[cdavernas]

@ricardozanini The pleasure is mine!

What about secured OpenApi endpoints? The runtime implementation should be able to handle those, IMO.

I am sorry but I fail to see how your example solves the auth problem. A single application executing such a call could very well be configured as you explained (and would still require restart on changes, right?), but I do not see how it could be applied to a workflow engine, which is domain-agnostic.

My concern is how can a domain-agnostic workflow engine be able to connect to a secured REST API endpoints it does not know anything about. As mentionned above, I know OpenApi defines a security description, which allows the workflow engine to understand what kind of auth scheme is used, and that is just great. But how do you proceed from there?

Say the endpoint is secured with the 'Bearer' scheme. The engine reads about it on the OpenApi spec, so it knows it should add an 'Authorization: Bearer {MYTOKEN}' header to the http request. From where would the engine retrieve that '{MYTOKEN}' value? Should it be specified as 'parameters' of the function call, in the workflow yaml/json, possibly colliding with 'real' parameters of the function?

And as for mounted secrets and/or CM in K8s, any new workflow definition with security requirements would require a pod restart, right (in the case I need to CRUD new variables/secrets)?

[cdavernas]

@tsurdilo First

Thanks for the repo, I'm already working on it. I'll need to add a couple of utils and extensions to the project, which are originating from my company's framework. I've no problem adding them, and even appending the license notice to them, but I need to be sure that there is no legal/copyright issue before that. I mean, can my framework, as well as existing or future products of my company still have those files without adding any serverless workflow-related copyright? Not that it's great code or sensitive stuff or whatever (which it obviously isn't), but I would prefer not jeopardizing whole libraries and/or projects because of that :)

Second, concerning the secrets, I absolutely agree with you, I'd rather avoid hard coded text-based secret files for obvious security concerns. I'm very excited to hear everyone's ideas on that subject.

[tsurdilo]

@Neuroglia Good question about licensing. Here is the doc that has all the details regarding ownerships to copyrights in CNCF projects: https://github.com/cncf/foundation/blob/master/copyright-notices.md Basically, if you contribute your code you still retain all the copyrights. So don't worry :)

Ideally you would use the same copyright headers as shown in the java and go sdk, but if not we can preserve the original headers and add: "Copyright The Serverless Workflow Specification Authors" (no need for the date).

I hope this helps put you at ease. This project is Apache 2.0 licensed and I don't see any issues with the question honestly. Let me know if you have any other concerns.

[cdavernas]

@tsurdilo Allright, many thanks, this puts me at ease indeed :)

[cdavernas]

@tsurdilo It looks like I do not have the authorization to push to the repo, even though I can edit files. Should I submit a PR instead?

[tsurdilo]

@Neuroglia yes please submit PR if possible

[tsurdilo]

@Neuroglia btw we have our meeting now (25 mins left) if you want to jump in a talk about this : https://zoom.us/my/cncfserverlesswg?pwd=YjNqYzhOdjRRd01YWFkzS1lHbDZqUT09

[ricardozanini]

@Neuroglia, sorry for the late reply. Too much is going on. Please find my replies inline.

What about secured OpenApi endpoints? The runtime implementation should be able to handle those, IMO.

But that's exactly what I said. I'm not sure if I could express myself correctly, but in the example I gave, the runtime is handling the secured endpoints. It reads from the OpenApi spec and generates code to call the secured endpoint based on it.

I am sorry but I fail to see how your example solves the auth problem. A single application executing such a call could very well be configured as you explained (and would still require restart on changes, right?), but I do not see how it could be applied to a workflow engine, which is domain-agnostic.

The engine is implementing the specification. So for every OpenApi definition found in the workflow file, it generates code to call the given endpoint - being secured or not.

My concern is how can a domain-agnostic workflow engine be able to connect to a secured REST API endpoints it does not know anything about. As mentionned above, I know OpenApi defines a security description, which allows the workflow engine to understand what kind of auth scheme is used, and that is just great. But how do you proceed from there?

Your engine must figure the calls itself :) In my case, I generated code to make such calls from the workflow engine.

Say the endpoint is secured with the 'Bearer' scheme. The engine reads about it on the OpenApi spec, so it knows it should add an 'Authorization: Bearer {MYTOKEN}' header to the http request. From where would the engine retrieve that '{MYTOKEN}' value? Should it be specified as 'parameters' of the function call, in the workflow yaml/json, possibly colliding with 'real' parameters of the function?

In the example I gave, the engine exposes system parameters/properties that can be passed during runtime via environment variables. Also, if deployed on a Kubernetes cluster, for example, one can bind a ConfigMap to it. You don't need to restart the pod to refresh such properties. After updating the ConfigMap, the runtime refreshes the properties automatically.

And as for mounted secrets and/or CM in K8s, any new workflow definition with security requirements would require a pod restart, right (in the case I need to CRUD new variables/secrets)?

Any new workflow definition would be a new deployment, so it won't require a restart but a fresh start. I'm not sure if I followed your point here.

Sorry if I misinterpret your use case, but I can't see how replicating the same information we already have on OpenApi in the Workflow can help here. I can see, though, that we might need to attach a reference to a secret or vault in an agnostic way to the workflow for any configuration we might need in the future. Runtimes can implement this "vault" as a Kubernetes secret, for instance.

But for this use case, I believe we are covered since it's just a matter of the runtime to expose such configuration to the platform.

[tsurdilo]

@ricardozanini @Neuroglia I am somewhat confused here. In most cases I assume that the openapi definition is a 3rd party def. If it is secured on an endpoint that we dont control, how can any runtime just know how to access it? Given that the user in the workflow definition defines the location of the json/yaml apiapi definition the runtime cannot know up-front what all possible locations and security means it will need to be able to access and use this definition.

I understand that maybe this is possible on some runtimes but I think we need some sort of generic approach to this. At first I think just to get by we do have ability to use the function definition metadata to add this info...but i think it's not a bad idea to add to the language a generic approach.

Note that our function definitions can also define gRPC definition resources, and not just OpenApi. And even to go further, we do allow function , event, and retry definition to be referenced via URI , which have nothing to do with OpenApi themselves. In those cases what do we do if those resources are also behind secured endpoints?

So I think we cannot just dismiss this request and should try to find out a solution for it.

[tsurdilo]

As far as secrets goes, I think we should start simple and grow from there. I don't think we can do what like step functions of google cloud workflow do where they have a custom IAM that can be configured for workflow resources.

Maybe we could start with env vars and them figure out what is missing and how to go from there? I do think that having access to env vars in expressions would be beneficial. WDYT?

[cdavernas]

@ricardozanini Thanks for your response, and no worries, I am myself overwhelmed with work ;)

So indeed, it seems I failed to explain my point of view, it will probably be much easier once my engine can be open-sourced. In the meanwhile, I think @tsurdilo understood perfectly my point, as shown in his last answer, especially regarding:

What about secured OpenApi endpoints? The runtime implementation should be able to handle those, IMO.

But that's exactly what I said. I'm not sure if I could express myself correctly, but in the example I gave, the runtime is handling the secured endpoints. It reads from the OpenApi spec and generates code to call the secured endpoint based on it.

To put it differently: How can you crawl the OpenApi document if it is itself secured? You have no way of knowing it ahead of time, and need to receive a 401/403 for realizing it. That's what I meant by secured OpenApi endpoint, as opposed to secured OpenApi operation, which can indeed be reflected upon thanks to the spec doc.

In the example I gave, the engine exposes system parameters/properties that can be passed during runtime via environment variables. Also, if deployed on a Kubernetes cluster, for example, one can bind a ConfigMap to it. You don't need to restart the pod to refresh such properties. After updating the ConfigMap, the runtime refreshes the properties automatically.

Ok, cool, I did not know I could update those at runtime. I'll try to run a single .NET container and see how can I play with its variables/secrets while running.

Any new workflow definition would be a new deployment, so it won't require a restart but a fresh start. I'm not sure if I followed your point here.

In my posts, I am speaking of a workflow engine, able to deploy and run workflow definitions without restart, in a similar fashion to Zeebe and Bonita BPM. I don't see why deploying a new definition would require a new deployment. Can you elaborate on that? IMHO, the only thing that should get deployed, if you want to leverage some of the great features of K8s, would be the definiton files themselves, as CRD.

[cdavernas]

Maybe we could start with env vars and them figure out what is missing and how to go from there? I do think that having access to env vars in expressions would be beneficial. WDYT?

@tsurdilo I don't know to who you last question was addressed, but if it was to me, the answer is cristal clear: YESYESYESYES!!!

[ricardozanini]

How can you crawl the OpenApi document if it is itself secured? You have no way of knowing it ahead of time, and need to receive a 401/403 for realizing it. That's what I meant by secured OpenApi endpoint, as opposed to secured OpenApi operation, which can indeed be reflected upon thanks to the spec doc.

Wait. You were talking about the resources we define in the definition to be secured? Oh my god, yes. I didn't understand the first time you come by.

Of course, our DSL must predict that. @tsurdilo I think every resource type we have that's a URI we need some way to define the information of how to access it. A good starting point is the schema from OpenApi.

In my posts, I am speaking of a workflow engine, able to deploy and run workflow definitions without restart, in a similar fashion to Zeebe and Bonita BPM. I don't see why deploying a new definition would require a new deployment. Can you elaborate on that? IMHO, the only thing that should get deployed, if you want to leverage some of the great features of K8s, would be the definiton files themselves, as CRD.

That depends on the engine. In my scenario, we generate the code upfront, so any modification to the definition we generate new code, which requires a new deployment. If only the configuration has changed, that won't need.

Maybe we could start with env vars and them figure out what is missing and how to go from there? I do think that having access to env vars in expressions would be beneficial. WDYT?

Sounds fine. Let's just try not to make things too wide. Generic ways of configuring can lead to a mess in the code. Like we discussed in the meeting, mixing infrastructure configuration and/or definition with business smells. :(

[tsurdilo]

thanks for all the inputs. i'll start working on adding env var support and then we can discuss the access part

[tsurdilo]

still working on this..didnt forget :)

[kevinteg]

For my understanding- is one of the goals of this spec to prevent leaking information about the runtime? If so, letting user-defined workflows access environment variables in expressions would be a potential issue.

[tsurdilo]

@cdavernas does https://github.com/serverlessworkflow/specification/pull/382 help with this a little? what else should be added?

[ricardozanini]

For my understanding- is one of the goals of this spec to prevent leaking information about the runtime? If so, letting user-defined workflows access environment variables in expressions would be a potential issue.

Hi @kevinteg, can you elaborate? Please see #382. We are adding an external configuration option to the DSL so that expressions can refer to them. I don't think we are leaking info about the runtime this way, but I can be missing something.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.