Add support for python appengine

[damienmg]

See bazelbuild/bazel#1067

[joeleu]

In an attempt to use bazel to run the hello_world application provided by Google, I receive the following error while using 'bazel run :main':

google.appengine.ext.builtins.InvalidBuiltinName: default is not the name of a valid builtin. Available handlers are: ERROR: Non-zero return code '1' from command: Process exited with status 1.

It appears there is an issue when parsing the builtins. None were provided in the app.yaml file but the dev_appserver is trying to add a 'default' builtin.

[ulfjack]

Adding @maximermilov, since commit ffe8c3fdc47d4ead45d02e908c56c21b1cc8967b says that it's adding support for Python AppEngine.

[maximermilov]

google.appengine.ext.builtins.InvalidBuiltinName:

I can't reproduce this. Did you change something in app.yaml?

[fejta]

Are the python app engine bazel rules supposed to work at this point? Do you have a working example for me to follow?

I am on the Kubernetes project and attempting to migrate to bazel how we build/deploy our gubernator app, which provides us with our PR dashboard as well as test results viewer.

However as best as I can tell these rules are either incomplete or else do not work with bazel 0.5.3

Any suggestions?

[Zaspire]

Are the python app engine bazel rules supposed to work at this point?

yes (appengine classic only).

However as best as I can tell these rules are either incomplete

Can you share branch with your build rules for gubernator app? I will help to debug problem.

[fejta]

Nice! Yes gubernator is a classic app (which as opposed to flex I assume?)

I will get you a branch with a python appengine app later tonight/tomorrow. I would prefer to start with a simpler, toy app rather than my full-blown app (what I want is a functional example).

In the mean time. our gubernator code is here (same link as before). Unfortunately there are no BUILD rules right now (I deleted my experimentation with creating them when the app engine rules didn't work).

The only docs, examples and tutorials I could find all assume I want to use java.

PS: I do not understand how this ROOT=$PWD line in the python deployment script is intended to work, as PWD can be anything when I call bazel-bin/gubernator/gubernator_deploy.sh. Is it supposed to find the .runfiles or something?

[fejta]

Here you go: https://github.com/fejta/test-infra/tree/gae/experiment/toy_app

gcloud app deploy works just fine however...

Repro:

git clone https://github.com/fejta/test-infra -b gae
cd test-infra
bazel build //experiment/toy_app && bazel-bin/experiment/toy_app/toy_app_deploy.sh
# bazel-bin/experiment/toy_app/toy_app_deploy.sh: line 12: .../test-infra/../com_google_appengine_python/appcfg: No such file or directory
bazel build //experiment/toy_app:toy_app.deploy && bazel-bin/experiment/toy_app/toy_app.deploy
# bazel-bin/experiment/toy_app/toy_app.deploy: line 12: .../test-infra/../com_google_appengine_python/appcfg: No such file or directory

[maximermilov]

&& bazel-bin/experiment/toy_app/toy_app_deploy.sh

instead should be bazel run experiment/toy_app:toy_app.deploy

Also:

• add application/version into app.yaml
• path to main.app should be relative to project root. i.e. "script: experiment.toy_app.main.app"

[fejta]

Thanks!

FYI, I removed the application/version tags from the app.yaml because this is what the cloud-sdk requires.

So the program now runs, woo! It appears to hang starting the admin server:

INFO: Running command line: bazel-bin/experiment/toy_app/toy_app
INFO     2017-07-31 17:18:59,178 sdk_update_checker.py:231] Checking for updates to the SDK.
WARNING  2017-07-31 17:18:59,413 simple_search_stub.py:1146] Could not read search indexes from /tmp/appengine.fejta-prod.fejta/search_indexes
INFO     2017-07-31 17:18:59,416 api_server.py:268] Starting API server at: http://localhost:32881
INFO     2017-07-31 17:18:59,419 dispatcher.py:199] Starting module "default" running at: http://localhost:8080
INFO     2017-07-31 17:18:59,420 admin_server.py:116] Starting admin server at: http://localhost:8000

Any further suggestions?

[fejta]

Oh got it, I need //experiment/toy_app:toy_app.deploy as the target

[maximermilov]

Oh got it, I need //experiment/toy_app:toy_app.deploy as the target

yes. Without ".deploy" suffix, it will run your app locally.

[joeleu]

I feel like I'm doing something very obviously wrong but cannot figure it out. I haven't changed any of the appengine files (app.yaml, main.py) but now it won't even build using the http_archive rule. bazel.zip

[jo2y]

Is anyone actively working on this? I'm working on a change to restructure the paths and filenames used to make things more consistent between the java and python implementations. That should make it easier to add other languages. Eg. https://github.com/bazelbuild/rules_appengine/issues/50

It's not going to be a non-backwards compatible change unless it's required. Eg. I've renamed appengine/appengine.bzl to appengine/java_appengine.bzl to match py_appengine.bzl. I'm also moving the files under test and examples into test/java and examples/java respectively.

One of the other style differences is how the appengine*.sh files are generated. For java, they are external files while python has them inline in the .bzl file. I've moved them to appengine/{java,py}/appengine*.sh.template. It's my personal preference to have them external because it keeps the .bzl file shorter and cleaner.

Anyone have any thoughts before I commit and send a PR?

[pmbethe09]

While I am not against restructuring, I am not clear what the gain is to rename to java_appengine.bzl versus the pain of the non-backwards compatible change. Perhaps consider moving the content of the file, but keeping a backwards compatible appengine.bzl which forwards you to java_appengine.bzl?

Moving the test/examples: seems fine

External vs. inline is likely hidden from the user and also backwards compatible.

You can send PRs to me.