Remote execution compatibility

Please is a client of the remote execution API however it has primarily been developed against a bespoke implementation of the servers, so compatibility with external server implementations is poor.

In this issue I will be tracking compatibility with the various major implementations.

General issues

Please requires some features from the as-yet unreleased v2.1 of the REAPI. Specifically, we need output_paths e.g. here

Some servers don't automatically store the outputs of a build action either. You may need to set the following in your .plzconfig:

[Remote]
VerifyOutputs = true

Please also performs dynamic actions post-build, so often needs to update action results. The build server has to allow this.

We also space separate the valid hashes for the remote asset API subresource integrity checks. This means any remote_file() rule that sets hashes = [] to a list with more than one item might break. This can be worked around with by passing --nohash_verification though this is not a production ready solution.

Otherwise, we are unaware of any other API compatibility issues, though there are surely more.

Testing setup

I used the docker-compose services defined in the remote-apis-testing repository to get the servers set up:

$ cd docker-compose
$ ./create_docker_compose_yaml.py
$ docker-compose -f docker-compose-buildbarn.yml up -d
$ docker-compose -f docker-compose-remote-assets.yml up -d

And then I can run please against this like so:

$ plz test  -o remote.url:localhost:8980 -o remote.asseturl:localhost:8979  -o remote.secure:false -o remote.instance:remote-execution -o remote.platform:OSFamily=linux -o remote.verifyoutputs:true

BuildBarn

Requires remote.verifyoutputs set to true on the Please side.

I have managed to build arcat against build barn so compatibility seems pretty good otherwise.

BuildGrid

TODO

buildbuddy

Doesn't support the output paths feature from the v2.1 API, and last i checked it's not a priority of theirs.

thought-machine / please