[Android] Put generateProto tasks as dependency for native build tasks

[serhiihuralniksc]

At the moment generateProto is not declared as dependency for tasks that are involved to generating and compiling native code. As result final build scripts (e.g. build.ninja) may lack rules for compiling protogenerated cpp.

[zhangkun83]

As I have little knowledge about compiling native code for Android, it would be helpful if you can point me to an example project that does so, and the names of the native build tasks.

[trevjonez]

I want to add a note here that when android studio syncs it runs gradle to get a model of the project via the tooling api's and then it try's to invoke the external native build directly to do indexing for the IDE. If your external native build depends on the generated output of this plugin being in place, android studio will fail to sync until you run the generation tasks from the CLI. The way studio fails is rather nondescript to the issue as well so it is hard to even realize what is happening.

My current workaround for this is the following:

task preSync

android.libraryVariants.all { variant ->
  preSync.dependsOn("generate${variant.name.capitalize()}Sources")
}

By adding a preSync task that will force a run of all variations of generateVariantNameSources and running that before try to sync studio we can ensure that all the proto files are in place for studio to do its external native build portion of the IDE sync process. In my case the time to run all those tasks is trivial but I do wish there was a way to have studio only use grade for 100% of the build concerns so it would be safely hookable. It is also bad/inefficient that we have to generate release source just to sync and work targeting a debug build variant etc...

[serhiihuralniksc]

@zhangkun83 sorry for delayed replay. at the moment I don't have sane minimal project. Currently we use next hack to get sources generated before external native build starts:

tasks.whenTaskAdded {
    if (it.name.startsWith('generateJsonModel')) {
        def variant = it.name.substring('generateJsonModel'.length())
        def genProtoTask = "generate" + variant + "Proto"
        it.dependsOn genProtoTask
    }
}

it appears that generateJsonModel* tasks are responsible for CMake run that generates final build system scripts (like build.ninja as I said earlier). All cpp sources must be in place when CMake runs, otherwise any globbing done in CMakeLists.txt will result in incomplete list of cpp stuff to be compiled. But being frankly even this approach sometimes doesn't work - protogenerated sources sometimes settle on disk only after CMake pass and linking ends up with undefined ref errors. Though this happens pretty rare, usually after sync project with remote repo or branch switching.

@trevjonez thanks for your solution. I think it may be easily adapted to get it working for CLI builds. One pros I see is that it shouldn't suffer from seldom issue I've described above.

[trevjonez]

I just figured out that you can use add_custom_command in cmake to tell it how to run gradle and fire the tasks for the files requred.

#PROTO_GEN_DIR and ROOT_PROJECT_DIR must be absolute for the files to be found correctly
add_custom_command(
        PRE_BUILD
        OUTPUT ${PROTO_GEN_DIR}/cpp/mytype.pb.cc
        OUTPUT ${PROTO_GEN_DIR}/cpp/mytype.pb.h
        DEPENDS ${CMAKE_SOURCE_DIR}/src/main/proto/mytypes.proto
        WORKING_DIRECTORY ${ROOT_PROJECT_DIR}
        COMMAND ./gradlew ${GRADLE_GEN_TASK} VERBATIM
)

then add ${PROTO_GEN_DIR}/cpp to your target_include_directories and ${PROTO_GEN_DIR}/cpp/mytype.pb.h and .cpp file to the add_library command.

and finally update your gradle script to pass the params down to the external cmake build:

android {
  defaultConfig {
    externalNativeBuild {
      cmake {
        arguments = [ /* Normal stuff here */
                     "-DROOT_PROJECT_DIR=${rootProject.projectDir.absolutePath}".toString()]
        /* flags, abiFilters */
      }
    }
  }
  buildTypes {
    release {
      minifyEnabled true
      proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
      externalNativeBuild {
        cmake {
          arguments += ["-DPROTO_GEN_DIR=${buildDir.absolutePath}/generated/source/proto/release".toString(),
                        '-DGRADLE_GEN_TASK=common:generateReleaseSources']
        }
      }
    }
    debug {
      externalNativeBuild {
        cmake {
          arguments += ["-DPROTO_GEN_DIR=${buildDir.absolutePath}/generated/source/proto/debug".toString(),
                        '-DGRADLE_GEN_TASK=common:generateDebugSources'] // or whatever task you have it hooked onto
        }
      }
    }
  }
}

and finally get rid of the presync task and just have external build depend on the generate sources task and any other task that needs to do env setup for you before running your build (npm install in my case)

android.libraryVariants.all { variant ->
  tasks.findByName("externalNativeBuild${variant.name.capitalize()}")
      .dependsOn(npmInstall, "generate${variant.name.capitalize()}Sources")
}

when the external build actually runs it will invoke gradle and get all up to dates for the tasks so maybe we could just have it echo nothing instead of actually invoking gradle? dare I make it fragile to save a few seconds?

[wangjiyang]

Any update on this issue?