novocaine / rust-python-ext

Distutils helpers for rust Python extensions
MIT License
68 stars 4 forks source link

Build/Install in Develop Mode? #15

Closed ghost closed 7 years ago

ghost commented 7 years ago

Hi, thanks for this module, it's a great step forward for Rust to have great python bindings, and the beginnings of a good distribution story.

I'm trying to integrate a rust module with a Python module using this tool, and one of the most uncomfortable things is: It doesn't seem to work in develop mode.

This is extra awkward because Rust is installed under my user but seems not to work with sudo when called by this lib. I suspect many other nightly-using rust devs will do likewise and install to user, not system.

Any hints as to how to get this running in develop mode to make the dev cycle a bit more comfortable?

novocaine commented 7 years ago

+1 I like develop mode too. What goes wrong currently? (sorry no time to check myself)

ghost commented 7 years ago

As mentioned in #16 I have been having confusing errors derived from using the PyPI version, which is outdated. So let me see if this works after all, in a few mins. :)

ghost commented 7 years ago

Right, so I've played around some more, and have some feedback.

For whatever reason, if I do sudo setup.py develop, the Python module will be built and linked, but the Rust lib is apparently not available. If I go directly to the build directory, I can load the build lib and the built rust-module, and they work well. But if I move to another directory and rely on Python's Path variable, it can't find the Rust lib.

If I do bdist or bdist_egg, it correctly stores the Rust lib in the tar file or egg file, and if I expand the file somewhere and visit the directory, I can load the files correctly. I'm not sure whether installation of these eggs would succeed, because the rust lib isn't mentioned in the TOP_LEVEL.txt file?

The problem is, I can't actually use sudo setup.py install because it tries to call cargo from root...which doesn't work if you installed Rust with rustup! Gorramn. Setuptools exposes a --skip-build option, is there some way to hook this for the rust building process? So, we could build rust modules as the user, and then do an install-only as root?

Summary: I can't seem to use develop mode, possibly some missing metadata issue in top_level.txt is leading to a link not being made. bdist options seem to package the lib correctly but the top_level.txt file still does not contain the additional top-level item for the Rust lib.

Maybe I'm using this wrong; perhaps there's some way to get the rust extension to be part of the Python module's source tree and namespace? But I can't see any way to make this happen, so it seems to be a parallel installation.. but it's not working that way for me.

novocaine commented 7 years ago

when you say it can't find the rust lib - what is the error you are receiving

fafhrd91 commented 7 years ago

I am not sure if it is appropriate, here my project https://github.com/fafhrd91/setuptools-rust it is based on this repo, but I added support for "develop" and "clean" and "bdist_wheel" commands

novocaine commented 7 years ago

added support for this in a3363caca63100a2fe24ec1bda8cbbaa4434c017

'develop' runs build_ext with inplace=1; inplace will put the .so in your package's dir so it's more likely be in the right path import-wise. so

  1. added a replacement develop cmdclass which does that
  2. supported inplace

in order to get it to work how you expect see my changes to https://github.com/novocaine/rust-python-ext/commit/a3363caca63100a2fe24ec1bda8cbbaa4434c017#diff-963a338046542f50c0a79b1c12412436 - you will probably want to come up with an ext_name that puts the extension in your package which has a module in it that is trying to import the extension

ghost commented 7 years ago

Thanks so much! I'll experiment with this later and see how it goes.

@fafhrd91: Looking forward to looking over your repo too, more options is always nice. :)

On 16 March 2017 23:16:26 GMT+00:00, James Salter notifications@github.com wrote:

Closed #15.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/novocaine/rust-python-ext/issues/15#event-1003669559

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.