search

JeremyBYU / bpyproj

A blender add-on for Map Projections
GNU General Public License v3.0
6 stars 1 forks source link

Handling Binary Dependencies for Pyproj #6

Closed JeremyBYU closed 6 years ago

JeremyBYU commented 6 years ago

I had a very difficult time trying to handle the binary dependencies for pyproj. At first I did the following:

  1. Install Pip using get-pip.py
  2. Then install pyproj using pip

This worked for many cases (meaning my personal machines), but invariably ran into many issues as soon as I began testing on blank slate virtual machines:

  1. The dependency requires a compilation process, at least for sure on linux. The linux system has to have c compiler AND has to have the python-dev package installed (needs Python.h and MANY other header files). Also these header files HAVE to be accessible from the python environment. So if you are using the embedded python environment built into Blender (Python 3.5), the header files need to be installed there!

Current Solution

Conda has a great package manager. They have prepackaged bundles for the module pyproj. The entire fully proper package including binary dependency for every Operating System (Windows, Linux, OSX), Architecture (32 bit vs 64bit), and mainstream python version (python 2.7, python 3.5, etc.) combination is readily available.

All new versions of blender use Python 3.5. I downloaded 64 bit versions of this package for all 3 main operating system and put them in the dependencies/binaries folder. My method then is to copy the module (along with its nicely packaged dependency) into the site-packages directory of the embedded python environment. So far testings has shown this to work really well, however there is still two issues

  1. Need to test on Mac OS (I believe the directory structure is different).
  2. Need to learn how to handle if Blender is not using the embdedd python environment. Its a little rare for this to occur, and is mostly only present if someone installed blender on linux using sudo apt-get install blender and already has python installed.
  3. Need to consider if I should just dynamically download the dependency (instead of including it in the git repository). This way I don't have to package them myself. I feel like this may be the better way, but maybe a combination of both is better (keep the three main OS's but download if the environment is different).
vvoovv commented 6 years ago

A Python package requiring compilation on Windows is nightmare. It requires installation of Microsoft Build Tools, handling missing header files, etc.

So I definitely like your approach.

We can require embedded Python only.

Just a short note. Could you print() what you download (including the full URL) and where you save the downloaded content to keep users informed?

JeremyBYU commented 6 years ago

Yeah I should be able to do that. Right now I am testing on my wifes MacBook Pro, but should be able to get to it.

JeremyBYU commented 6 years ago

Okay that should be working now. I print the location for where I download the files from and where they are installed to.

Do you agree that I should bundle the dependency (pyproj, binary included) inside the git repository to make things simpler. The dependency is 5MB each (not too big), and I am supporting 3 operating systems.

vvoovv commented 6 years ago

Just tried to test interaction of the addons. It didn't download anything. EDIT: the code handling download is simply not there.

I'd prefer to simplify the things wherever possible. So not to run Blender under administrator on Windows, avoid possible network problems, etc.

So lets bundle dependencies with the bpyproj addon!

JeremyBYU commented 6 years ago

I have NOT started the download code yet. I also have NOT bundled the dependencies yet, they are in the .gitignore file. But seeing that you agree it simpler to distribute them, I will remove the .gitignire line and now upload them to GitHub.

On Wed, May 2, 2018, 8:03 AM Vladimir Elistratov notifications@github.com wrote:

Just tried to test interaction of the addons. It didn't download anything.

I'd prefer to simplify the things wherever possible. So not to run Blender under administrator on Windows.

So lets bundle dependencies with the bpyproj addon!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/JeremyBYU/bpyproj/issues/6#issuecomment-385954590, or mute the thread https://github.com/notifications/unsubscribe-auth/AGUi9TZlA8IWFyBvCCBZ1Ne5xJju-GI4ks5tuaCBgaJpZM4TufgB .

JeremyBYU commented 6 years ago

Okay just pushed the dependencies up to GitHub for all three major Operating Systems. However there was one important thing to note. You said:

I'd prefer to simplify the things wherever possible. So not to run Blender under administrator on Windows

There is unfortunately nothing I can do about this. Almost all blender programs will be installed in C:\Program Files and the corresponding Python environment will be in this directory as well. This is offlimits to move files (install pyproj) without Administrator privilege. But if people are hesistant about giving Blender admin privileges for this install, the documentation (README.md) explains that they can look at the log file which will tell them how to manually copy the files themselves. It provide the full source and destination paths.