search

aws / aws-lc-rs

aws-lc-rs is a cryptographic library using AWS-LC for its cryptographic operations. The library strives to be API-compatible with the popular Rust library named ring.
Other
236 stars 40 forks source link

Consolidate FFI bindings #371

Open Ralith opened 3 months ago

Ralith commented 3 months ago

aws-lc-sys exposes separate bindings for every supported target. These files are huge, totaling almost 8MiB today and growing to 14MiB after #369. They are also almost identical to eachother: e.g. diff -U3 linux_aarch64_crypto.rs linux_x86_64_crypto.rs reveals that the only difference is the representation of the va_list argument to OPENSSL_vasprintf.

Most Rust FFI bindings are able to support diverse platforms using a single in-tree binding file (perhaps generated by bindgen with allowlisted symbols), leaving platform-specific types to be supplied by std::os::raw and/or libc, or at worst a small hand-written helper module. Can aws-lc-sys do similar?

justsmth commented 3 months ago

Thanks for your feedback. I agree that these bindings files are huge, and their size grows linearly with the number of platforms that we pre-generate them for.

Although the size of the bindings will nearly double for 0.14.0, the bindings are just a small part of the crate contents. From my testing, the size of the aws-lc-sys crate will be increasing from ~6.8 MB compressed (~45 MB uncompressed) for 0.13.3 to ~7.5 MB compressed (~52 MB uncompressed) for 0.14.0. So, this will roughly be a 15% increase in the overall crate size.

We are open to ideas or suggestions on how to improve our bindings or improve the way they're generated. Unfortunately, considering the size of the AWS-LC API and the fact that it's still evolving, it would be too laborious and error-prone to maintain the bindings manually. Is there a tool that might automate the consolidation of very similar Rust source files (like linux_aarch64_crypto.rs and linux_x86_64_crypto.rs) in a way that preserved their platform-specific differences? That would be great!

justsmth commented 3 months ago

One possibility is to parse the various Rust bindings files, consolidating the types/values common to all into a single source file, leaving only what's unique to a platform in its respective platform-specific file that would be include!-ed.

justsmth commented 3 months ago

Simple change to help reduce space -- we could disable the "layout tests" for the bindings we publish in the crate: https://github.com/aws/aws-lc-rs/blob/main/aws-lc-sys/builder/bindgen.rs#L120

camshaft commented 3 months ago

Seems like we're including the libssl symbols as part of the pregenerated bindings as well. That seems unnecessary, especially since the primary purpose here is a libcrypto replacement.