search

google / asylo

An open and flexible framework for developing enclave applications
https://asylo.dev
Apache License 2.0
967 stars 126 forks source link

Cannot build enclave reproducibly #41

Closed tiziano88 closed 4 years ago

tiziano88 commented 4 years ago

I cannot seem to reproducibly build the enclave code on my machine. I am using the following steps:

  1. docker run -it --rm -v bazel-cache:/root/.cache/bazel -v "${PWD}":/opt/my-project -w /opt/my-project gcr.io/asylo-framework/asylo
    • [execute the following steps from within the running Docker container]
  2. bazel build --config=sgx-sim //asylo/examples/hello_world
  3. cp bazel-bin/asylo/examples/hello_world/hello_enclave_unsigned.so enclave_$(date --iso-8601=seconds).so
  4. bazel clean
  5. bazel build --config=sgx-sim //asylo/examples/hello_world
  6. cp bazel-bin/asylo/examples/hello_world/hello_enclave_unsigned.so enclave_$(date --iso-8601=seconds).so
  7. sha1sum enclave_*

Note that the resulting hashes differ, e.g. on my machine I get the following:

b88b0156ebc354ef15cf706fdf4428e998b68356  enclave_2019-10-01T18:51:50+00:00.so
e48909687b68f586e1c3b4facf66cad2b4e51f1a  enclave_2019-10-01T18:55:54+00:00.so

ref https://github.com/project-oak/oak/issues/241 cc @anghelcovici @deeglaze @KeithMoyer @annasapek

tiziano88 commented 4 years ago

FWIW I analyzed the difference between the two resulting binaries with diffoscope, and it seems most (all?) of the differences are around functions such as:

From a quick look at the repository, these seems to be defined in https://github.com/google/asylo/blob/5ab253e2db786a9cccbc397aac2338531d4aa119/asylo/platform/system_call/type_conversions/manual_types_functions.cc .

KeithMoyer commented 4 years ago

@tiziano88, thanks for the good bug report with simple reproducer! I've root caused this and we'll be working on a fix (it should be fairly simple).

We have some code generation for translating between constant values when marshaling across the enclave boundary, and the python dictionary used at one stage of this is itself non-deterministic in ordering. This results in the order these functions are generated to be different, which breaks reproducability. Sorting the dictionary keys (and thus the outputted function order) should fix the issue you're seeing.

tiziano88 commented 4 years ago

Thanks for the fix @carmenyh , I have tried pulling in the latest version of Asylo including your fix, but I am still seeing inconsistent results. The difference seem still related to the ordering of symbols such as:

I suggest reopening this issue until the root cause is identified.

KeithMoyer commented 4 years ago

@tiziano88, this was automatically closed via commit description annotation prematurely. Reopening