markmc / rebuilding-the-wheel

1 stars 2 forks source link

rust dependency management #80

Open dhellmann opened 4 months ago

dhellmann commented 4 months ago

The build isolation works for packages that use Python and C, but the rust built tools expect to download libraries during the build process, still.

tiran commented 4 months ago

I borrowed the keys to the time machine and requested a feature for the problem three years ago in setuptools-rust issue 112. With the 'new' feature, setuptools-rust can automatically vendor Rust crates into the sdist. I also tried to convince upstream projects like PyCA cryptography to vendor their crates, but upstream was against it.

Vendoring Rust crates after the fact is a bit complicated. Also setuptools.build_meta is stubborn and requires extra convincing to bundle stuff. Also re-created of sdists may be missing some files that were added by setuptools_scm unless MANIFEST.in also includes them.

# /tmp/setup-extra.cfg
[sdist]
vendor_crates=1
$ curl https://files.pythonhosted.org/packages/e6/4c/77135117237c4ef579da90d51a38defb50a332f57e5c94663bad9c793744/maturin-1.5.1.tar.gz | tar xz

$ cd maturin-1.5.1

$ DIST_EXTRA_CONFIG=/tmp/setup-extra.cfg python3 -m build -s

$ $ ls -sh dist/maturin-1.5.1.tar.gz
44M dist/maturin-1.5.1.tar.gz

The vendored tar ball is huge, because it contains tons of unnencessary Windows API bindings. My vendor_rust.py script from Fedora's python-cryptography dist-git has the same problem.

Our colleagues from CoreOS had the same issue and created https://github.com/coreos/cargo-vendor-filterer . Something like this works for maturin. For other projects, we have to path the maninfest file explicitly.

$ cargo install cargo-vendor-filterer
$ SOURCE_DATE_EPOCH=0 cargo vendor-filterer --platform=x86_64-unknown-linux-gnu --prefix=vendor --format=tar.gz

then we need to create a .cargo/config similar to the cargo_prep macro in CentOS/RHEL:

$ rpm --eval "%cargo_prep -V 1"
(
/usr/bin/mkdir -p .cargo 
cat > .cargo/config << EOF 
[build]
rustc = "/usr/bin/rustc"
rustdoc = "/usr/bin/rustdoc"

[env]
CFLAGS = "-O2 -g"
CXXFLAGS = "-O2 -g"
LDFLAGS = "%{build_ldflags}"

[install]
root = "/root/rpmbuild/BUILDROOT/%{NAME}-%{VERSION}-%{RELEASE}.x86_64/usr"

[term]
verbose = true
EOF
# %if 01
/usr/bin/tar -xoaf %SOURCE1
cat >> .cargo/config << EOF 

[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "./vendor"
EOF
# %endif
)