search

pypa / auditwheel

Auditing and relabeling cross-distribution Linux wheels.
Other
443 stars 145 forks source link

mangling and dependency over .so in other wheels #410

Open amol- opened 1 year ago

amol- commented 1 year ago

I tried to come up with an experiment to understand how a big package containing many shared objects could be divided in smaller packages all containing some shared objects and allowing each wheel to rely on the shared objects provided by the other wheel.

It seemed to me that --exclude could have allowed that when the python libraries are imported in the proper order, so I created https://github.com/amol-/wheeldeps

If my understanding is correct, the mangling of shared objects would prevent that experiment from being successful, because once auditwheel repair is applied, one package would provide libfoo-ef63151d.so (the package where libfoo is embedded), but the other package would look for libfoo.so as it had --exclude libfoo.so and thus would not recognise libfoo-ef63151d.so as the library that it is looking for.

What would be the best way to handle this use case?

amol- commented 1 year ago

I was able to make my proof of concept work correctly by applying the name mangling done by auditwheel to the excluded library too ( https://github.com/amol-/wheeldeps/commit/6b1f8af703309a1fa4d017a139f6c74cc9793792 ).

But that seems like an hack and not something that might be a long term solution.

amol- commented 1 year ago

I built https://github.com/amol-/consolidatewheels as a way to address this problem and consolidate mangling convention across multiple wheels. This is now also used in https://github.com/amol-/wheeldeps/blob/main/try.sh#L9 experiment to consolidate wheels and make the experiment work.

If anyone needs the same approach, this is now available on pypi https://pypi.org/project/consolidatewheels/