Automatically publish the precompiled artifacts on each release

[fhoering]

We are starting to deploy the key/value server on our own infra to do some experiments.

Currently we see the architecture as having a C++ key/value runtime that rarely changes and having an independent Javascript UDF project that can be released with the runtime.

It means the Javascript UDF project should not have to checkout and compile this github repo every time.

It would be very useful if the github repo could automatically publish the build artifacts for supported architectures. Currently only sources files are published (see https://github.com/privacysandbox/protected-auction-key-value-service/releases/tag/v0.16.0). Precompiled binaries could be added here. It could be also used to publish the code hashes for the TEE attestation part as this needs to be verified by the KMS/Coordinator (https://github.com/privacysandbox/protected-auction-services-docs/blob/main/key_value_service_trust_model.md)

On our side we are interested in those binaries:

• ./server
• ./udf_delta_file_generator
• ./data_cli

We use the Ubuntu build for now and amd64 & arm64 architectures.

[peiwenhu]

This is on our radar but there's internal legal processes to satisfy before we can do this. Would it be helpful in the meantime if we provide a cloud build script (AWS/GCP) that you can use to run on AWS or GCP to build it?

[fhoering]

We would really like to deploy this on our own infra to assess how easy it is and audit the dependencies.

Also scripts like ./udf_delta_file_generator & ./data_cli are provided as c++ code with this project but are actually independent of the runtime workflow and will not be executed in the cloud TEE.

[peiwenhu]

Oh I was just suggesting that we can provide a script to build the artifacts on cloud. Once built, you can run it anywhere you like, assuming we provide them as docker images.

[fhoering]

OK. But I would also need the ubuntu based artifacts not only the GCP/AWS specific artifacts.

If it is hosted in github or on a cloud web server matter less, at least in a first step. It could be a web page also like for other open source project releases https://spark.apache.org/downloads.html

Don't forget that the web server that hosts those artifacts could also include the container code hashes that are needed to verify the attestation. So being able to trust this web server seems important.

[emmafu2022]

Hi, FYI we have uploaded the following docker images to https://github.com/privacysandbox/protected-auction-key-value-service/releases/tag/v0.16.0.

1. amd_local_server_docker_image.tar (for running server locally)
2. amd_tools_binaries_docker_image.tar (for data_cli tool)

The build rule for local server docker image can be found here. And here is the example command to run the server locally in docker:

docker run -it --init --rm --volume=/data:/data --security-opt=seccomp=unconfined bazel/production/packaging/local/data_server:server_docker_image -- --port 50051

The document for how to run data_cli tool in docker image can be found here

[thegreatfatzby]

Hey @emmafu2022 is the idea here that we should be able to run a basic instance of the image locally using the images you guys have created. That would be really great...do we need permissions, or am I doing something wrong here:

Isaacs-MacBook-Pro:protected-auction-key-value-service isaacfoster$ docker login
Authenticating with existing credentials...
Login Succeeded
Isaacs-MacBook-Pro:protected-auction-key-value-service isaacfoster$ docker run -it --init --rm --volume=/data:/data --security-opt=seccomp=unconfined bazel/production/packaging/local/data_server:server_docker_image -- --port 50051
Unable to find image 'bazel/production/packaging/local/data_server:server_docker_image' locally
docker: Error response from daemon: pull access denied for bazel/production/packaging/local/data_server, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

[emmafu2022]

Hi @thegreatfatzby did you load the docker image before running the docker run command? docker load -i /amd_local_server_docker_image.tar

[fhoering]

Hi, FYI we have uploaded the following docker images to https://github.com/privacysandbox/protected-auction-key-value-service/releases/tag/v0.16.0.

OK. Thanks @emmafu2022. It looks good to me. I didn't test yet but we will test the images on GCP & AWS.

My understanding in the long run would be that those images can be directly released as a TEE to the cloud platform and are then accepted by the coordinator.