How to run the program in linux

[tonyng1707]

Hi, I am new in python. May I ask how to run the program after downloading the source on Linux? Thanks

[takluyver]

I don't know if this still works. It was an experiment which didn't really go anywhere, and I stopped working on it about 18 months ago.

If you want to play with it anyway, cd into the source tree, then:

pip install flit
flit install

Now you should have a wheel2conda command available, which you can run as shown in the README.

[jph00]

@takluyver I've been planning to create something similar - just found this. Can I ask why it didn't go anywhere? Any "gotchas" I should know about, if I try going down a similar path myself?

[takluyver]

I think it technically worked well enough for pure-Python wheels. I didn't try to make it work for packages with compiled libraries, and that would probably be a much tougher nut to crack - I can expand on why if that's what you're thinking of.

One issue I do recall is that there was no documentation of the conda package format, at least in 2016. It was easy enough to inspect some existing packages and produce compatible ones, but no documentation implies no guarantee that the format is stable.

The main reason it didn't go anywhere was just that no-one really wanted it. At the time, conda-forge co-ordinated three different CI services to build on different platforms, and particularly the Mac builds sometimes had to queue for days. I thought that a shortcut for pure Python packages would be valuable - but special casing them in this way was a non-starter. Since then, free CI has got more capable, and conda has gained 'noarch' support, which allows many pure Python packages to be built on a single platform. So there's less pressure now to do something like this.

[Aside: I see from a quick stalk that you're one of the founders of Fastmail. I'm now a happy user, so :wave: :+1:, even if you've moved on to other things]

[jph00]

You have excellent taste in email providers! :) Also, I'm a fan of your bash kernel, which works really well with nbdev and fastpages for documenting terminal stuff.

I found this which looks like it might be helpful https://pypi.org/project/setuptools-conda/

Please do tell me the issues you found with wheels and compiled code.

One key thing I want wheel2conda for is that conda packages can't depend on pip packages. So I need an easy way to move pip packages onto a conda channel.

[takluyver]

Thanks, that's good to hear. :-)

https://pypi.org/project/setuptools-conda/

I haven't seen that before. Glancing at the code, it looks like it's actually calling conda-build internally to build the package from source. That's more moving parts, but probably a more robust approach than this, where I tried to bypass conda's own build machinery and create the packages directly.

Please do tell me the issues you found with wheels and compiled code.

I didn't actually try this, but I'd expect problems if you tried to turn binaries from wheels directly into conda packages. Published wheels typically bundle non-Python libraries they depend on - e.g. an h5py wheel bundles libhdf5. In conda, hdf5 can be packaged itself, so the h5py package will depend on it. I guess it could work to put the bundled dependencies in the conda package as well, but it's a pretty weird situation for conda to manage, and I wouldn't be surprised if problems crop up.

Compiled extension modules can also link against libraries in other Python packages (in particular, a lot of scientific Python packages will link against numpy). You may need some extra work to ensure that the translated packages depend on a compatible version of the package they're linked against.

So I need an easy way to move pip packages onto a conda channel.

I'd probably look at setuptools_conda or similar approaches - wrapping the normal conda-build machinery for making conda packages. Between this and noarch packages (pure Python packages built once for all platforms), it can probably be pretty convenient, and building on the standard tools makes it easier to do something extra where a more complex package needs it.

[jph00]

I've actually already created something that creates conda package files directly, bypassing conda-build. It's way faster, and easier for a simpleton like me to understand. I find the errors from conda-skeleton and conda-build hard to understand and fix, so creating the tgz directly is something I much prefer. Also conda modifies the libs and stuff in ways that often break things in pre-built binaries, and I never managed to find a way to fully avoid that - but creating the tgz directly means I know exactly what I end up with.

I'm not too fussed about stuff with complex dependencies - it's simple largely-standalone things I'm most interested in packaging.

[takluyver]

Nice! I'm also a fan of creating stuff directly, at least where it's for my own use - but I hesitate to recommend it to anyone else, especially where the format is undocumented. Maybe it's been documented since I was looking, though.

[beenje]

I was pointed out to this repo and find the concept really interesting. We have many pure python packages internally at work. This would be an easy way to create noarch conda packages. @jph00, did you end up doing something similar or not?

[jph00]

I found various useful bits and pieces - you can see stuff I used here: https://github.com/fastai/fastconda/tree/master/.github/workflows

[beenje]

I found various useful bits and pieces - you can see stuff I used here: https://github.com/fastai/fastconda/tree/master/.github/workflows

Thanks!

[tibdex]

You can give python-wheel-to-conda-package a try ;).