search

cdrx / docker-pyinstaller

PyInstaller for Linux and Windows inside Docker
MIT License
618 stars 241 forks source link

requitements txt with external dependencies cannot be satisfied #32

Open deeagle001 opened 6 years ago

deeagle001 commented 6 years ago

Hello,

First thank you for the great work! :)

Second, I have found a theoretical problem in your image.

I try to build wxpython based application with your images, and with Windows it works fine. However, when I try to do the same, I have GTK missing error for the wxpython listed in the requirements.txt file.

You provided an example in the Readme.md how to overcome this, but it cannot be fixed with that.

The example you provided in the Readme.md actually not replaces the entrypoint, just passes it as an argument, which will be executed as the install step.

So if there is a third party dependency (wxpython->GTK3), there is no way to satisfy it till the time the requirements.txt processed.

I know two workaroud here:

I consider both of them as a hack, and there should be some way to install third party dependencies before requirements.txt processed without rebuilding the image.

jameshilliard commented 6 years ago

Are you using the windows binary wheel installer for wxpython? I think you need to make sure all your packages in requirements.txt that have native extensions use binary wheels since docker-pyinstaller doesn't have the necessary MSVC compilers present to build them from source.

I've used this successfully to cross compile a wxpython based app for windows after ensuring my requirements.txt used binary wheel installers for all packages with native extensions. There were a few packages I had to compile myself in a windows 7 VM. You'll probably need to maintain a separate requirements.txt for each platform as the dependencies will often be slightly different.

Here's what my wxpython app requirements.txt looks like for windows(I uploaded my pre-compiled whl packages to github releases for convenience):

altgraph==0.14
appdirs==1.4.3
asn1crypto==0.23.0
attrs==17.2.0
autobahn==17.9.3
Automat==0.6.0
bcrypt==3.1.4
boto3==1.4.7
botocore==1.7.34
https://github.com/jameshilliard/bsdiff4/releases/download/1.1.4/bsdiff4-1.1.4-cp36-cp36m-win_amd64.whl
certifi==2017.7.27.1
cffi==1.11.2
chardet==3.0.4
constantly==15.1.0
cryptography==2.1.1
defusedxml==0.5.0
docutils==0.14
dsdev-utils==0.9.6
https://github.com/jameshilliard/python-ed25519/releases/download/1.4/ed25519-1.4-cp36-cp36m-win_amd64.whl
future==0.16.0
h2==3.0.1
hpack==3.0.0
hyperframe==5.1.0
hyperlink==17.3.1
idna==2.6
incremental==17.5.0
jmespath==0.9.3
JMS-Utils==1.0.2
macholib==1.8
netaddr==0.7.19
paramiko==2.3.1
pbr==3.1.1
pefile==2017.9.3
priority==1.3.0
pyasn1==0.3.7
pyasn1-modules==0.1.5
pycparser==2.18
PyInstaller==3.3
PyNaCl==1.1.2
pyOpenSSL==17.3.0
pypiwin32==220
pyserial==3.4
python-dateutil==2.6.1
PyUpdater==2.5.0
PyUpdater-s3-Plugin==3.0.6
PyUpdater-SCP-Plugin==3.0.12
requests==2.18.4
s3transfer==0.1.11
scp==0.10.2
service-identity==17.0.0
six==1.11.0
SOAPpy==0.12.22
stevedore==1.27.1
https://github.com/jameshilliard/treq/zipball/HTTPDigestAuth
https://github.com/jameshilliard/twisted/releases/download/twisted-17.9.0/Twisted-17.9.0-cp36-cp36m-win_amd64.whl
txaio==2.8.2
urllib3==1.22
wstools==0.4.5
wxPython==4.0.0
zope.interface==4.4.3
deeagle001 commented 6 years ago

Thank you for the answer! Actually, the Windows build works like a charm. Because as you mentioned, the windows wheel contains the dependency (x86, x86-64).

I have a problem with the linux build, as there is where the GTK3 is missing.

This is a specific case with the wxpython, but my problem is generic. You cannot install third-party tools before pip execution, to satisfy the requirements.txt dependencies.

I would propose another optional environment variable, which can contain commands to install third-party dependencies before parsing the requirements.txt. This would make your solution much more generic and could solve many complex problems.

jameshilliard commented 6 years ago

The linux builds probably need to be overhauled to use static libraries for the build so that they are portable as I mentioned here.

cdrx commented 4 years ago

If you wanted to make a PR that adds some functionality to install system packages before installing requirements.txt I would gladly merge that