Feature Request

[ahmet8282]

it would be great if the location of the state files are somewhere on the container. Then i can just map the directory to S3, or local volume. And stated become persistent without any batch jobs.

So for example, if you save the model files in

/models

on the docker container, then with docker-compose, I can make that directory path persisted to a volume. so the contents of the directory gets saved and reloaded by docker

[dahernan]

That could work for local development, but I don't see how will work on kubernetes or on google appengine flexible.

Is any way that you can expand the example, how that could work on AWS?

[ahmet8282]

i can explain it for kubernetes easier.

For example Redis. The following is the yaml file. It uses a persistent volume that is mounted to /data directory. And this directory is where Redis keeps instance data. So even if we remove the container, and create it again, it just continues to use the files in the same directory. Same example applies for sql servers. They keep their data in a directory, and that directory is mounted from a persistent volume of the cloud provider. In a way, whaever file is saved to this directory, gets permanently stored elsewhere.

Aws specific instructions are here: https://kubernetes-on-aws.readthedocs.io/en/latest/user-guide/using-volumes.html

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  revisionHistoryLimit: 0
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:4.0.11-alpine
        ports:
        - containerPort: 6379
        volumeMounts:
          - name: redis
            mountPath: /data
      - name: goproxy
        image: k8s.gcr.io/goproxy:0.1
        ports:
        - containerPort: 6379
        readinessProbe:
          tcpSocket:
            port: 6379
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 6379
          initialDelaySeconds: 15
          periodSeconds: 20
      volumes:
      - name: redis
        persistentVolumeClaim:
          claimName: redis-data

[dahernan]

ok, I think I've got it.

The main problem all the models are memory base, and save to disk is expensive. But we can think about it, and see if it make sense

[ahmet8282]

In the end, right now the developer is expected to call an api to download the model. And it will never be current for the developer either, as it is a live system. So even if you make it save to disk every once in a while, would help a lot.

[matryer]

I personally don't think we should add this functionality to the box, since you can solve it via the API. The more things the box is doing, the more things can go wrong, so we tend to be very tight on what gets in. @ahmettahasakar Do you need some help implementing this kind of functionality yourself?

[dahernan]

Closing this for now, thank you for the suggestion