PyPi *Source* and Wheel Distributions Contain Architecture-Specific Binaries and Possible GPLv2 License Violation

danieldjewell commented 4 years ago

Hello!

Both the source and binary wheel distributions of pyexiv2 (as of v2.3.1 as hosted on PyPi) contain binaries compiled for x86-64. When installing on aarch64, for example, the package is completely broken.

Couple of things:

Source distributions should never contain binaries - that's the whole point of source distributions... even for external libraries.
Binary wheels with the "none-any" platform tag are not expected to contain architecture dependent code .... e.g. pyexiv2-2.3.1-py3-none-any.whl would be an architecture independent wheel. It's perfectly fine to have pre-compiled binaries in a binary wheel but they need to be properly tagged. See https://packaging.python.org/specifications/platform-compatibility-tags/
Not using the binary packaging guidance has resulted in unnecessary bloat in the form of including extraneous files that are unneeded for the current OS, current architecture, and current python version. The Packaging standards were designed to (mostly) address this. That's why there are tags for different OSes, Architectures, and Python Versions. The directory listing below shows 11MiB installed - when at most it should be ~2.8mb per platform.

.
├── [211K]  darwin64-py35
│   └── [207K]  exiv2api.so
├── [211K]  darwin64-py36
│   └── [207K]  exiv2api.so
├── [211K]  darwin64-py37
│   └── [207K]  exiv2api.so
├── [212K]  darwin64-py38
│   └── [208K]  exiv2api.so
├── [2.6M]  exiv2.dll
├── [2.6M]  libexiv2.dylib
├── [3.4M]  libexiv2.so
├── [221K]  linux64-py35
│   └── [217K]  exiv2api.so
├── [221K]  linux64-py36
│   └── [217K]  exiv2api.so
├── [222K]  linux64-py37
│   └── [218K]  exiv2api.so
├── [215K]  linux64-py38
│   └── [211K]  exiv2api.so
├── [299K]  win64-py35
│   └── [295K]  exiv2api.pyd
├── [299K]  win64-py36
│   └── [295K]  exiv2api.pyd
├── [300K]  win64-py37
│   └── [296K]  exiv2api.pyd
└── [304K]  win64-py38
    └── [300K]  exiv2api.pyd

11M used in 12 directories, 15 files

Possible GPLv2 License Violation of exiv2

Exiv2 is released under the GPLv2 -- which AFAIK requires that if you distribute binaries, you must make the source code available for those binaries. Further, this is the right thing to do on many fronts. Check out: https://reproducible-builds.org

Further, read GNU's GPL guidance on this very topic here: https://www.gnu.org/licenses/gpl-faq.html#UnchangedJustBinary

Making binaries is fine - but you have to release the source. Since exiv2 is already on github ... adding it as a git submodule and then having python build the dependency would be the best option. That way, if someone downloads the package on a different architecture, they will be able to build it too :grin:

References

https://www.python.org/dev/peps/pep-0425/

LeoHsiao1 commented 4 years ago

Thank you for your warning.

About binaries:
- I have not considered architecture such as aarch64. Because I'm just renting some VPS on the cloud platform as a build environment.
- The binary version of pyexiv2 is not compatible with different platforms, or even with different versions of the Python interpreter. To avoid user compilation failures, I manually compiled the commonly used versions.
- When compiling pyexiv2 on Windows, it need to install Visual Studio, which means I have to pre-compile pyexiv2 for Windows users.
After I packaged pyexiv2 with python setup.py, it was automatically named pyexiv2-xxx-none-any.whl. In the future, I can split up pyexiv2-xxx-py3-none-any.whl into multiple files for different platforms.
I'll add exiv2 as a git submodule.

dgtlmoon commented 3 years ago

Isn't including binaries a horrible security issue?

LeoHsiao1 commented 3 years ago

In v2.4.1 , I finally released the wheel package for different platforms separately. So they've been reduced in size. All compiled files have been removed from the source package.

Isn't including binaries a horrible security issue?

The source code package does not contain compiled files, while the wheel package contains compiled files. Next, I'll automate packaging and publishing pyexiv2 through GitHub Actions. So you don't have to worry about me tampering with the compiled files.

github-actions[bot] commented 3 years ago

This issue has been automatically closed because there has been no activity for a month.

LeoHsiao1 / pyexiv2