Google App Engine deployments?

[igordertigor]

I see that so far, mlem can only deploy to heroku natively. For google app engine, there is no dedicated deployment type. What's the timeline for adding something like that? Right now it seems that the workaround would be to run mlem serve inside the app.yaml. Is that correct? Is there any plan to provide an adapter for this?

[aguschin]

Hi @igordertigor! We're going to release K8s, SageMaker and "deploy" to Docker containers very soon (they're already available in release/0.3.0 branch). Google App Engine is not on our agenda yet. How does the deployment to Google App Engine looks like in your case? Do you use pre-built Docker image, or a boostrap script in some fixed environment that installs everything + downloads the model?

[igordertigor]

Hi @aguschin , thanks for the quick reply. I haven't actually deployed an mlem model to GAE yet, but was planning to use the pre-built Docker image if possible. I believe that something like this would work for the app.yaml

# Default runtime is python38
runtime: python38
# Default instance class is F1 (set depending on model)
instance_class: F1

entrypoint: mlem serve $MODEL_SPEC

env_variables:
    MODEL_SPEC=<INSERT FROM GITHUB ACTIONS>

(obviously, mlem should be in the requirements.txt file). One nice feature of the GAE Basic Environment is that it scales to zero and is quite cost efficient for extremely low workloads, which makes it particularly attractive for prototypes.

[aguschin]

Do you know MLEM can build docker images that run mlem serve inside? https://mlem.ai/doc/get-started/deploying If you don't need anything except your model there, it may be a good option.

[igordertigor]

Of course I know. Unfortunately, running docker in GAE requires the non-standard tier, which requires a tiny bit more maintenance from our side. We would prefer to avoid that in favour of the basic GAE environment. But then, the deployment script isn't super complex anyway. It would just be nice to streamline everything with mlem. If you could give me some pointers where to start, I would be happy to take a look into how much effort it would be to implement it and potentially just submit a pull request.

By the way congrats on the new release.

[mike0sv]

I took a quick look and it seems that something like this can prepare a directory with all that you need

import os
import sys

from yaml import safe_dump

import mlem
from mlem.api import load_meta, build
from mlem.contrib.fastapi import FastAPIServer
from mlem.core.objects import MlemModel
from mlem.runtime.server import Server

def prepare_for_gae(path: str, server: Server, target: str = ".", project: str = None, rev: str = None, ):
    model = load_meta(path, project, rev, force_type=MlemModel)
    os.makedirs(target, exist_ok=True)
    model.clone(os.path.join(target, "model"))
    reqs_path = os.path.join(target, "requirements.txt")
    build("requirements", model, target=reqs_path)
    with open(reqs_path, "a") as f:
        reqs = server.get_requirements() + [f"mlem=={mlem.__version__}"]
        print(reqs)
        f.write("\n".join(reqs.to_pip()))
    with open(os.path.join(target, "app.yaml"), "w") as f:
        f.write(f"runtime: python{sys.version_info[0]}{sys.version_info[1]}\n"
                f"entrypoint: mlem serve --load server.mlem --model model")
    with open(os.path.join(target, "server.mlem"), "w") as f:
        f.write(safe_dump(server.dict()))

def main():
    prepare_for_gae("model", server=FastAPIServer(), target="./build")

if __name__ == '__main__':
    main()

However I did not find a good way to then deploy this to GAE from code. They have this library https://googleapis.dev/python/appengine/latest/index.html but docs are practically non-existant. Also didn't find any good app.yaml specification :(

If you can help with questions like this:

• How to create GAE application/service from code?
• How to get service status from code?
• How to deploy new version of the app from code?
• How to remove app from code?

then I will be able to help you create a full MlemDeployment implementation based on snippet above

[igordertigor]

Hi @mike0sv, thank you for sharing your script. I believe that I was doing something quite similar but without using mlem internals. Regarding your questions, I'll assume that the user is logged into the target gcloud project (looking at at least the heroku deployment, that seemed to be assumed there too). Then:

1. Call gcloud app deploy inside a folder with an app.yaml file, will upload everything in that folder to GAE and deploy to the app that's specified in app.yaml as "service". If no service is specified, it defaults to "default". If the target service doesn't exist, it will be created. Otherwise a new version will be created. Tricky here: The first service that you deploy will always be the "default".
2. A little tricky. Three commands might be useful
   • gcloud app describe describes the full GAE system of the respective project in yaml. There is a field servingStatus. However, that refers to the full GAE environment and not an individual component.
   • gcloud app services describe <service name> does something similar but for an individual service (e.g. the deployed mlem model). Unfortunately, there is no servingStatus field here.
   • gcloud app logs tail shows the logs tails the logs for the "current" (accoring to local app.yaml service. So the best option appears to parse the logs, I believe.
3. gcloud app deploy will create a new version of the existing app and also try to route traffic to the new version. More elaborate stuff (e.g. traffic splitting to different services) would go through gcloud app services <subcommand>. But I don't know very much about that part.
4. gcloud app services delete <service name> deletes the specified service.

And yes, there is very little and confusing/outdated documentation. I hope this helps at least a bit.

[mike0sv]

Is there a way to do this without calling gcloud in subprocess?

[mike0sv]

This page might be helpful https://cloud.google.com/appengine/docs/admin-api/deploying-apps And also https://googleapis.github.io/google-api-python-client/docs/dyn/appengine_v1.html whatever it might be

[igordertigor]

Is there a way to do this without calling gcloud in subprocess?

Not an easy way that I'm aware of. But as you already pointed out: It's kind of a mess with the docs.

You could of course assemble the requests yourself. I think you already pointed there with the second comment.

[mike0sv]

It's just hard to ensure that gcloud is installed. Eg if sometime in the future we will want to assemble docker containers that can manage deployments, there will be no easy way to install it. I'll try to see if it can be done with requests

[igordertigor]

That certainly makes sense. It might be tricky to get the credentials in that case though.

[mike0sv]

I think there is an env var with path to json key file. It can be used to make those oauth calls (or whatever gcloud uses). Also we can find where gcloud stores it's credentials and use them. Basically it's what aws do inside boto3 for example, the difference is that they have slightly better docs :)

[mike0sv]

Mother of God, gcloud is written in python2.7 in 2013. I'm literally scared of what I might discover next

Ok App Engine part is written in 2016

[igordertigor]

Oh ha!