Can a function mount a volume to load configs and data?

[derailed]

I am trying to figure out how one could mount a volume either using PVC or Configmap. Looking at the sample functions I am not finding a good example on how to best achieve this. Any pointers would be much appreciated!

UseCase: I need to define a function that can configure itself or load static data prior to being invoked. I think leveraging k8s resource such as mounting a volume from a ConfigMap/Secret or PV/PVC would be a great addition if not supported. The fn.yml could surface similar manifest mechanic as it k8s counterparts ie volume and volumeMounts.

Expected Behaviour

Add ability to mount a volume so that a function can init and configure itself prior to serving requests

Current Behaviour

Can't seem to find an example??

Possible Solution

Leverage k8s volume/volume mount manifest resources

Steps to Reproduce (for bugs)

1. 2. 3. 4.

Context

I have a set of functions that need higher level configuration vs using plain old env vars. I'd like to be able to inject configmaps/volumes in order to configure my functions. In K8s I can setup configmap/pvc resources and linked then in my function yml definition.

Your Environment

• Docker version docker version (e.g. Docker 17.0.05 ):

• Are you using Docker Swarm or Kubernetes (FaaS-netes)? Kubernetes

• Operating System and version (e.g. Linux, Windows, MacOS): Linux/OSX

• Link to your project or a code example to reproduce issue:

[alexellis]

Hi Fernand,

From what I'm reading I think you're suggested several things:

1. mount arbitrary volumes
2. mount configs (config files)

There is no support for either of these points, however storage is available through an S3 interface - here is a detailed tutorial.

Point 1 is addressed here https://github.com/openfaas/faas/issues/320 and there is no stance on point 2 at this stage although since we have support for secrets already config maps for (FaaS-netes) and Swarm configs for (FaaS-Swarm) may be worth further investigation.

Can you tell me more about an actual use-case for the following? It will help us understand the requirement instead of the suggested implementation. I'd also like to understand if we already provide ways to achieve the same result(s) with the current design.

I need to define a function that can configure itself

What does this mean? What is a real example?

Or load static data prior to being invoked

Please can you provide a use-case for this too.

[stefanprodan]

Hi @derailed

Mounting files or volumes inside a function is not yet supported.

Setting config data can be done via environment variables and secrets. Working with files can be done using Minio. You would create a secret with your Minio address and auth and when your function starts you would connect to Minio, download any files your function needs form a Minio bucket, process them and delete them before existing.

[derailed]

@alexellis @stefanprodan - Thank you so much for the prompt reply and for all your efforts and awesome contributions. I am a big fan!!

I'll totally understand if this is an edge case request and if this is not at all on the radar!

NOTE: Could you point me to resources where I can understand the various fields supported for the fn.yml? Can't seem to locate good docs??

I understand, I can specify configs via env vars but it feels totally Jurassic to me and would rather load from a CM if possible.

UseCase: Self Configuring Functions

Say I have a function. I would like to configure how this function will run based how it's invoked or perhaps a single function definition could result into several function pods being launched with different configuration, yet still manage via a single k8s service. Multiplexing across the various pods could be handled via routing/headers/etc...

This would provide an affordance to compare different implementations of a given handler for performance tuning or service configuration ie training a neural net and need to see outcomes based on given set of factors ie facialRecogFn is more/less precise using config1 vs config2

Say I have a config map that specifies the following yaml

algorithms:
  v1: 
     opt1: 42
     opt2: 10.5 
     ...
  v2:
     opt1: 42.75
     opt2: 11.0
     ...

I understand that I could deploy 2 functions for this and use different handler. But I am thinking reuse. ie keep the handler pure and hence the Docker image and inject configuration to tune behavior is better!

Having the ability to inject this config into my function and then setting an env to configure the function runtime ie VERSION=v1|v2|... would be very useful IMHO. The scenario I am envisioning is training a corpus for a neural net and be able to tune the results based on these settings. For this use case. I think it would be useful to deploy several instances of the same function running different algs and be able to specify which function to run either via header/cookie/route params/etc...

I have an Istio based OpenFaas cluster I am currently working on and though this would absolutely be a killer feature as one could comb traffic and hence behavior based on initial impetus. So I am thinking canary functions support if that makes sense. I have this think up and running but it's way more complicated that it needs to be for this particular use case.

Most of the FAAS impls I've looked at seems to offer 1-n svc to fn where functions are just setup as cloned replicas. What if these weren't just clones? and one could tune behavior? I think having the ability to fire off different handler/image of the same function or change behavior based on configs for each function pods, would be an awesome addition as one could opt out a given implementation or yield different function behavior based on a given impetus.

Anyhow this is rather long winded, but this is what I am trying to achieve for my cluster and hope we can figure out a way to work together to make this a reality!

Then again "Fernand, you need to lay off the pipe!" Would also be a completely acceptable response...

[stefanprodan]

You could create a secret that contains your config like this:

kubectl create secret generic facial-recog --from-file=./config.yaml

Read the config in your function code from this file:

/run/secrets/facial-recog/config.yaml

In order to switch algorithms you can call your function using a query string:

http://localhost:8080/function/facial-recog?algorithm=v1

Read the algorithm version in your function code from the Http_Query environment variable:

Http_Query=algorithm=v1

You can read here how the query stings and HTTP headers are transformed into env vars.

[derailed]

Awesome!! Thanks @stefanprodan for this information. Much appreciated!

[ericstoekl]

Derek close: initial question was answered, further specific questions should be raised as separate issues.

[alexellis]

Most of the FAAS impls I've looked at seems to offer 1-n svc to fn where functions are just setup as cloned replicas. What if these weren't just clones? and one could tune behavior? I think having the ability to fire off different handler/image of the same function or change behavior based on configs for each function pods, would be an awesome addition as one could opt out a given implementation or yield different function behavior based on a given impetus.

I am not sure that the replicas of a function should differ - that may be better accomplished with two separate deployments of the function (could use the same function (read Docker) image though with your config/secret mounted/bound.

Istio / Nginx etc could then sit in front and re-route as necessary. Istio does look very versatile for re-routing traffic - can it do this with configurable routes too?

[derailed]

Thank you all for inputs!

You are right @alexellis. I think this is indeed the correct approach as you do want to keep the ability to scale/autoscale a function. I do think it would be a good thing to have the ability to introduce a new function say f' that can use f docker image and somehow inject configuration to tune behavior ie training functions f and f'. So in the K8s use case, having the ability to inject a CM or PV I think would be a good thing imho.

Best I can tell is Istio can totally handle the routing, but it must do so at the service level. Thus one does need to introduce a new service to front f and f'. This is a very straight forward think to do as you can specify custom labels in the functions .yaml and have the service select on those or possibly use a headless service too...

So my initial incline on this one is wrong. I think these are indeed 2 separate functions as you've pointed out Alex. I do think however that this use case would be best served if one could use CM or volumes to inject configurations and keep the image clean so it could be reused across functions??

[rmb938]

Going back to the original question, I do like the idea of being able to give configmaps to functions. An example would be a shared config across various functions. So instead of setting env variables manually in each function definition I can just set a configmap name similar to secrets. For now I can use secrets however that is not really the "kubernetes" way to do things.

[pyramation]

can somebody elaborate (or show an example yaml) what method they are using to get config maps and secrets into the function?

I'm finding information all over - but nothing really clear. I found the secrets docs on the website but not for configmaps?

e.g.

provider:
  name: faas
  gateway: http://127.0.0.1:8080

functions:
  query-db:
    lang: node
    handler: ./query-db
    image: query-db:latest
    env:
    - name: POSTGRES_USER
      valueFrom:
        configMapKeyRef:
          name: airpc-postgres-config
          key: app.db.user
    - name: POSTGRES_PASSWORD
      valueFrom:
        configMapKeyRef:
          name: airpc-postgres-config
          key: app.db.pass

[alexellis]

Derek lock

[alexellis]

Hi @pyramation

Secrets are documented in the workshop https://github.com/openfaas/workshop and via the docs https://docs.openfaas.com/

If after consulting both you are still having problems then please feel free to raise your own issue.