mit-plv / fiat-crypto

Cryptographic Primitive Code Generation by Fiat
http://adam.chlipala.net/papers/FiatCryptoSP19/FiatCryptoSP19.pdf
Other
701 stars 144 forks source link

Fiat-Crypto: Synthesizing Correct-by-Construction Code for Cryptographic Primitives

Building

CI (Coq, Docker) CI (Coq, Debian) CI (Coq, Alpine) CI (Coq, Arch Linux) CI (Coq, Windows) CI (Coq, MacOS)

Release Zulip Rust Crate Go Reference

🌐 Try out synthesis on the web! (Supported by js_of_ocaml compilation.)

This repository requires Coq 8.18 or later. Note that if you install Coq from Ubuntu aptitude packages, you need libcoq-ocaml-dev in addition to coq. Note that in some cases (such as installing Coq via homebrew on Mac), you may also need to install ocaml-findlib (for ocamlfind). The extracted OCaml code for the standalone binaries requires OCaml 4.08 or later. We suggest downloading the latest version of Coq. Generation of JSON code via the Makefile also requires jq.

Alternatively, choose your package-manager to install dependencies:

Package Manager Command Line Invocation
Aptitude (Ubuntu / Debian) apt install coq ocaml-findlib libcoq-ocaml-dev jq
Homebrew (OS X) brew install coq ocaml-findlib coreutils jq
Pacman (Archlinux) pacman -S coq ocaml-findlib ocaml-zarith jq
APK (Alpine) apk add --repository=https://dl-cdn.alpinelinux.org/alpine/edge/testing ocaml ocaml-findlib coq ocaml-zarith jq make

You can clone this repository with

git clone --recursive https://github.com/mit-plv/fiat-crypto.git

Git submodules are used for some dependencies. If you did not clone with --recursive, run

git submodule update --init --recursive

from inside the repository. Then run:

make

You can check out our CI to see how long the build should take; as of the last update to this line in the README, it takes about 1h10m to run make -j2 on Coq 8.11.1.

If you want to build only the command-line binaries used for generating code, you can save a bit of time by making only the standalone-unified-ocaml target with

make standalone-unified-ocaml

Usage (Generating C Files)

The Coq development builds binary compilers that generate code using some implementation strategy. The parameters (modulus, hardware multiplication input bitwidth, etc.) are specified on the command line of the compiler. The generated C code is written to standard output.

A collection of C files for popular curves can be made with

make c-files

The C files will appear in fiat-c/src/.

Just the compilers generating these C files can be made with

make standalone-ocaml

or make standalone-haskell for compiler binaries generated with Haskell, or make standalone for both the Haskell and OCaml compiler binaries. The binaries are located in src/ExtractionOcaml/ and src/ExtractionHaskell/ respectively.

There is one compiler binary for all implementation strategies:

This binary takes arguments for the strategy:

Passing no arguments, or passing -h or --help (or any other invalid arguments) will result in a usage message being printed. These binaries output C code on stdout.

Here are some examples of ways to invoke the binaries (from the directories that they live in):

# Generate code for 2^255-19
./fiat_crypto unsaturated-solinas '25519' '64' '5' '2^255 - 19' carry_mul carry_square carry_scmul121666 carry add sub opp selectznz to_bytes from_bytes > curve25519_64.c # 1
./fiat_crypto unsaturated-solinas '25519' '32' '10' '2^255 - 19' carry_mul carry_square carry_scmul121666 carry add sub opp selectznz to_bytes from_bytes > curve25519_32.c # 2

# Generate code for NIST-P256 (2^256 - 2^224 + 2^192 + 2^96 - 1)
./fiat_crypto word-by-word-montgomery 'p256' '32' '2^256 - 2^224 + 2^192 + 2^96 - 1' > p256_32.c # 3
./fiat_crypto word-by-word-montgomery 'p256' '64' '2^256 - 2^224 + 2^192 + 2^96 - 1' > p256_64.c # 4

Try out the above on the web 🌐1 🌐2 🌐3 🌐4. You can find more examples in the Makefile.

Note that for large primes, you may need to increase the stack size to avoid stack overflows. For example:

ulimit -S -s 1048576; ./fiat_crypto word-by-word-montgomery --static gost_512_paramSetB 32 '2^511 + 111'

This sets the stack size to 1 GB (= 1024 MB = 1024 * 1024 KB = 1048576 KB) before running the command. As of the last edit of this line, this command takes about an hour to run, but does in fact complete successfully. Without a sufficiently large stack-size, it instead stack overflows.

Usage (Generating Bedrock2 Files)

Output is supported to the research language bedrock2. The Coq development builds binary compilers that generate code using some implementation strategy. The parameters (modulus, hardware multiplication input bitwidth, etc.) are specified on the command line of the compiler. The generated bedrock2 code is then written to standard output using the bedrock2 C backend.

A collection of bedrock2/C files for popular curves can be made with

make bedrock2-files

The bedrock2/C files will appear in fiat-bedrock2/src/.

Just the compilers generating these bedrock2/C files can be made with

make standalone-ocaml

or make standalone-haskell for binaries generated with Haskell, or make standalone for both the Haskell and OCaml binaries. The binaries are located in src/ExtractionOcaml/ and src/ExtractionHaskell/ respectively.

There is one compiler binary for all implementation strategies:

This binary takes arguments for the strategy:

Passing no arguments, or passing -h or --help (or any other invalid arguments) will result in a usage message being printed. These binaries output bedrock2/C code on stdout.

Here are some examples of ways to invoke the binaries (from the directories that they live in):

# Generate code for 2^255-19
./bedrock2_fiat_crypto unsaturated-solinas --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select '25519' '64' '5' '2^255 - 19' carry_mul carry_square carry_scmul121666 carry add sub opp selectznz to_bytes from_bytes > curve25519_64.c # 1
./bedrock2_fiat_crypto unsaturated-solinas --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select '25519' '32' '10' '2^255 - 19' carry_mul carry_square carry_scmul121666 carry add sub opp selectznz to_bytes from_bytes > curve25519_32.c # 2

# Generate code for NIST-P256 (2^256 - 2^224 + 2^192 + 2^96 - 1)
./bedrock2_fiat_crypto word-by-word-montgomery --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select 'p256' '32' '2^256 - 2^224 + 2^192 + 2^96 - 1' > p256_32.c # 3
./bedrock2_fiat_crypto word-by-word-montgomery --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select 'p256' '64' '2^256 - 2^224 + 2^192 + 2^96 - 1' > p256_64.c # 4

# Generate code for 2^130 - 5
./bedrock2_fiat_crypto unsaturated-solinas --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select 'poly1305' '64' '3' '2^130 - 5' > poly1305_64.c # 5
./bedrock2_fiat_crypto unsaturated-solinas --no-wide-int --widen-carry --widen-bytes --split-multiret --no-select 'poly1305' '32' '5' '2^130 - 5' > poly1305_32.c # 6

Try out the above on the web 🌐1 🌐2 🌐3 🌐4 🌐5 🌐6. You can find more examples in Makefile.examples.

License

Fiat-Crypto is distributed under the terms of the MIT License, the Apache License (Version 2.0), and the BSD 1-Clause License; users may pick which license to apply.

See COPYRIGHT, LICENSE-MIT, LICENSE-APACHE, and LICENSE-BSD-1 for details.

Reading About The Code

Status of Backends

Fiat Cryptography contains a number of backends; the final step of the pipeline is transforming from straight-line C-like code to expressions in whatever language is being targeted. The Bedrock2 backend comes with proofs that the Bedrock2 AST matches the semantics of our internal AST, but none of the other backends have any proofs about them. While the transformations are not particularly involved, and our proofs ensure that we have picked integer sizes large enough to store values at each operation, there is no verification that the particular integer size casts that we emit are sufficient to ensure that gcc, clang, or whatever compiler is used on the code correctly selects integer sizes for expressions correctly. Note that even the C code printed by the Bedrock2 backend does not have proofs that the conversion to strings is correct.

Hence we provide here a table of the extent to which the various backends are maintained, tested, and proven. A :heavy_check_mark: in "maintainer" means that the Fiat Cryptography maintainers are fully maintaining and testing the backend; :white_check_mark: means maintenance by external contributors. We do not provide any quality guarantees for code generated by the backends.

Backend Maintainer / Person of Contact Build Checked by CI Generated Code Tested Internal AST Proven Correct
C :heavy_check_mark: @JasonGross / entire team :heavy_check_mark: :heavy_check_mark: (BoringSSL test-suite)
Bedrock2/C :heavy_check_mark: @andres-erbsen / entire team :heavy_check_mark: :heavy_check_mark: (BoringSSL test-suite) :heavy_check_mark:
Go :white_check_mark: @mdempsky :heavy_check_mark:
Java :x: Unmaintained :heavy_check_mark: :x: Known Buggy #707
JSON Experimental :white_check_mark: (only syntax)
Rust :white_check_mark: :heavy_check_mark: :heavy_check_mark: (Dalek Cryptography test-suite)
Zig :white_check_mark: @jedisct1 :heavy_check_mark: :white_check_mark: (Zig standard library (generated file here) (main CI here))

Contributing a new backend

We weclome new backends (as long as you're willing to maintain them). We hope that the process of contributing a new backend is not too painful, and welcome feedback on how the process could be streamlined. To contribute a new backend, follow the following steps (perhaps using, for example, #936, #660, #638, or #570 as examples):

Reading The Code

Demo of Synthesis

The idea of the synthesis process is demoed in src/Demo.v. We strongly recommend reading this before studying the full-scale system.

Proofs About Elliptic Curves

We have some about elliptic curves, for example:

Actual Synthesis Pipeline

The entry point for clients of the PHOAS expressions we use is Language/API.v. Refer to comments in that file for an explanation of the interface; the following text describes how the expressions are generated, not how to interact with them.

The ordering of files (eliding *Proofs.v files) is:

Language/*.v
    ↑
    β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
AbstractInterpretation/*.v     MiscCompilerPasses.v    Rewriter/*.v     PushButtonSynthesis/ReificationCache.v      Arithmetic.v
    ↑                                ↑                       ↑                       ↑                                   ↑
Stringification/*.v                  β”‚                       β”‚                       β”‚                        COperationSpecifications.v
    ↑                                β”‚                       β”‚                       β”‚                                   ↑
    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                   β”‚
           BoundsPipeline.v                                  CompilersTestCases.v                                        β”‚
                 ↑                                                                                                       β”‚
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     PushButtonSynthesis/*.v
                              ↑
                   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                  CLI.v                SlowPrimeSynthesisExamples.v
                   ↑
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
StandaloneHaskellMain.v   StandaloneOCamlMain.v
        ↑                           ↑
ExtractionHaskell.v          ExtractionOCaml.v

Within each directory, the dependency graphs (again eliding *Proofs.v and related files) are:

Within Language/:

  Pre.v ←──────────────────────────────────────────────────────────────────────── IdentifierParameters.v
    ↑                                                                                        ↑
Language.v ←── IdentifiersBasicLibrary.v ←──── IdentifiersBasicGenerate.v ←── IdentifiersBasicGENERATED.v ←───────────────────────────── API.v
    ↑                        ↑                                                               ↑
    β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       └────────────────────────────┐                                  β”‚
UnderLets.v    IdentifiersLibrary.v ←──────────── IdentifiersGenerate.v ←─────── IdentifiersGENERATED.v
                     ↑                                       ↑                               ↑
              IdentifiersLibraryProofs.v ←─── IdentifiersGenerateProofs.v ←─ IdentifersGENERATEDProofs.v

Within Stringification/:

Language.v
    ↑
   IR.v
    ↑
 β”Œβ”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”
C.v       Rust.v

We will come back to the Rewriter/* files shortly.

The files contain:

The files defined in Rewriter/ are split up into the following dependency graph (including some files from Language/ at the top):

IdentifiersLibrary.v ←───────────────────────── IdentifiersGenerate.v ←──────────────────── IdentifiersGENERATED.v
    ↑ ↑                                                   ↑                                        ↑
    β”‚ └──────────────── IdentifiersLibraryProofs.v ←──────┴─ IdentifiersGenerateProofs.v ←─ IdentifersGENERATEDProofs.v
    β”‚                                     ↑                                                        ↑
    β”‚                                     β”‚                                                        β”‚
    β”‚                                     β”‚                                                        β”‚
    β”‚                                     β”‚                                                        β”‚
    β”‚                                     β”‚                                                        β”‚
Rewriter.v ←────────────────────── ProofsCommon.v ←──────────────────── ProofsCommonTactics.v      β”‚
    ↑                                 β†—        β†–                                ↑                  β”‚
Reify.v ←──────────────┐           Wf.v   InterpProofs.v                        β”‚                  β”‚
                       β”‚              β†–        β†—                                β”‚                  β”‚
Rules.v                └──────────── AllTactics.v β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                  β”‚
    ↑                                      ↑       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
RulesProofs.v                         AllTacticsExtra.v
    ↑                                      ↑
    β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
    β”‚   Passes/NBE.v    Passes/Arith.v    Passes/ArithWithCasts.v    Passes/StripLiteralCasts.v
    β”‚        ↑             ↑                        ↑                             ↑
    β”‚        └─────────────┴────────────────────────┴─────────────────────────────┴─────────────┐
    β”‚                                                                                           β”‚
    └────────┬──────────────────────────┐                                                       β”‚
      Passes/ToFancy.v      Passes/ToFancyWithCasts.v                                           β”‚
             ↑                          ↑                                                       β”‚
             β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     β”‚
                   All.v

Proofs files: For Language.v, there is a semi-arbitrary split between two files Language.Inversion and Language.Wf.