Unable to deploy flex application with enable_app_engine_apis: true as WAR

[justindriggers]

Migrating from the Managed VM java-compat runtime, following the instructions here: https://cloud.google.com/appengine/docs/flexible/java/upgrading#continuing_to_use_compat_runtimes

src/appengine/app.yaml:

service: my-application
runtime: java
env: flex

handlers:
- url: /.*
  script: this field is required, but ignored
runtime_config:  # Optional
  jdk: openjdk8
  server: jetty9
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 3.35
beta_settings:
  enable_app_engine_apis: true
health_check:
  enable_health_check: true
  check_interval_sec: 50
  timeout_sec: 30
  unhealthy_threshold: 10
  healthy_threshold: 5
  restart_threshold: 60

After running appengineDeploy, the following error is reported:

Starting Step #0
Step #0: INFO  - Identified Java artifact for deployment Artifact{type=WAR, path=/workspace/my-application.war}
Step #0: Exception in thread "main" com.google.cloud.runtimes.builder.buildsteps.base.BuildStepException: App Engine APIs have been enabled. In order to use App Engine APIs, an exploded WAR artifact is required, but a WAR artifact was found. See https://cloud.google.com/appengine/docs/flexible/java/upgrading for more detail.
Step #0:    at com.google.cloud.runtimes.builder.buildsteps.RuntimeImageBuildStep.getBaseRuntimeImage(RuntimeImageBuildStep.java:76)
Step #0:    at com.google.cloud.runtimes.builder.buildsteps.RuntimeImageBuildStep.run(RuntimeImageBuildStep.java:51)
Step #0:    at com.google.cloud.runtimes.builder.BuildPipelineConfigurator.generateDockerResources(BuildPipelineConfigurator.java:96)
Step #0:    at com.google.cloud.runtimes.builder.Application.main(Application.java:105)
Finished Step #0

Since the plugin detects that the application is a "flex" application, there is no explodeWar task, so I whipped one up:

appengine {
    stage {
        artifact = "$buildDir/exploded-app/"
    }
}

task explodeWar(type: Sync) {
    into appengine.stage.artifact
    with war
}

assemble.dependsOn explodeWar

However, that only results in an error complaining about the artifact not being a file:

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task 'appengineStage'.
> File '${project_path}/my-application/build/exploded-app' specified for property 'stagingConfig.artifact' is not a file.

I believe that has to do with the @InputFile annotation on StageFlexibleExtension#getArtifact() (@Input supports files and directories).

To work around this, we are currently exploding the WAR between the appengineStage and appengineDeploy tasks:

appengine {
    stage {
        stagingDirectory = "$buildDir/staged-app/"
    }
}

task cleanupWar(type: Delete) {
    delete fileTree(appengine.stage.stagingDirectory) {
        include '**/*.war'
    }
}

task explodeWar(type: Copy) {
    into appengine.stage.stagingDirectory
    with war
    finalizedBy cleanupWar
}

appengineStage {
    finalizedBy explodeWar
}

This way, the WAR is built and copied into the staged-app directory, and then exploded directly inside of the staged-app directory (while cleaning up the WAR so that it doesn't get built into the Dockerfile). Using this setup, the appengineDeploy task correctly deploys the exploded WAR with the enabled APIs using the flex runtime.

Ideally, I think the appengineStage task should simply detect the presence of enable_app_engine_apis: true in the app.yaml, and explode the WAR on its own.

[patflynn]

@meltsufin since this is related to builders. @lesv @ludoch

@justindriggers Thanks for the bug report. I don't think we had every considered the use-case of a flex app with no appengine-web.xml using the standard compat APIs. @lesv is that a supported configuration?

If so I think @meltsufin will have to change the builder validation to allow use of the app engine APIs from apps not using the standard framework.

[meltsufin]

WAR deployment to compat is not supported. You have to use an exploded WAR.

[patflynn]

@meltsufin that makes sense if the standard framework (appengine-web.xml and staging) is required. Is that the case?

[meltsufin]

Yes, appengine-web.xml is required as well.

[patflynn]

@lesv if @meltsufin is correct then that means the documentation that @justindriggers referenced is incorrect. It suggests the user can use an app.yaml directly to use the compat apis which is in contradiction with what Mike has stated.

[justindriggers]

To be clear, we have successfully deployed an application with the flex runtime with enable_app_engine_apis: true and an app.yaml file using the method described toward the end of my first post:

appengine {
    stage {
        stagingDirectory = "$buildDir/staged-app/"
    }
}

task cleanupWar(type: Delete) {
    delete fileTree(appengine.stage.stagingDirectory) {
        include '**/*.war'
    }
}

task explodeWar(type: Copy) {
    into appengine.stage.stagingDirectory
    with war
    finalizedBy cleanupWar
}

appengineStage {
    finalizedBy explodeWar
}

It is important that we can access the AppEngine APIs from the flex runtime in the same manner that we could using the old vm: true configuration.

[patflynn]

@meltsufin it sounds like we need to remove the builder validation that requires an exploded war to enable app engine apis.

[meltsufin]

AFAIK App Engine Flex Compat has always required an exploded WAR with appengine-web.xml and the builder is designed around that assumption. You can go with runtime: custom and do non-standard things there. Otherwise, please file an issue on the builder.

[lesv]

Agree - docs are wrong. b/69170708

[justindriggers]

Thanks for the info, I'll try to get it working with the appengine-web.xml and report back sometime next week.

[justindriggers]

Switching to using the appengine-web.xml with <env>flex</env> appears to work without any tricks or workarounds in Gradle. It looks like it's compiling a war with the XML file included, then extracting it automatically, and the appengineStage task is converting the XML into its equivalent YAML format.

A couple of peculiar things I've noticed:

1) apply plugin: 'com.google.cloud.tools.appengine' seems to be applying the AppEngineStandardPlugin rather than the AppEngineFlexiblePlugin, even though the environment is set to "flex". This makes navigating the documentation pretty confusing (I'm deploying a Flex app, but I should be using the Standard documentation?). 2) The resulting app.yaml is formatted slightly differently than what the documentation shows:

beta_settings:
  'enable_app_engine_apis': 'true'
  'source_reference': 'de43e7c6edf690c96ff3934ae77894ea35917d3d'
api_version: '1.0'

as opposed to:

beta_settings:
  enable_app_engine_apis: true

In any case, my original issue is resolved, but feel free to keep it open for tracking any documentation changes.

[loosebazooka]

flex-compat follows the appengine standard workflow (appengine-web.xml, exploded-war, etc) which is why you are seeing AppEngineStandardPlugin applied. <env>flex</env> just tells the deployment that the target environment is flex-compat.

If you were to deploy a pure-flex style app, then AppEngineFlexiblePlugin would be applied.

[justindriggers]

Should I be setting my <env> to flex-compat then? Just using flex seems to be working.

[loosebazooka]

Sorry, I appear to have just confused you more. flex is fine.

[justindriggers]

No worries, thanks for the clarification.