Allow other ways to install cadquery

[rowanG077]

I have had nothing but trouble with conda. Everytime I have to reconfigure my machine I basically have to take a day to get conda working. Is there a possibility to move away from it and just starting using pypi?

[adam-urbanczyk]

I don't think it is realistically an option. Pip does not really support binary depandancies like oce. What is exactly the issue with conda? It works flawlessly on ci so I'm surprised that you are having difficulties with it. Would providing a docker image or a custom anaconda installer help?

[rowanG077]

I think I have installed conda 5 times in total just to use cadquery. Out of those 5 times only once it worked out of the box. One of the times was on windows where it currently still isn't working. On my current OS conda doesn't work at all which means I currently run a Ubuntu VM just to be able to use cadquery. Conda to me is incredibly brittle software so I would really just prefer a normal environment.

[adam-urbanczyk]

@rowanG077 sorry to hear that. If you are willing to share what does not work I could maybe help. I had success following those instructions: https://docs.anaconda.com/anaconda/install/silent-mode/

[ilya-epifanov]

@adam-urbanczyk The problem with conda is that it pollutes the environment. I just removed it (again) because it was breaking everything else python-related on my machine in very pervert ways.

Docker might be A solution to this. It's should be much easier to use for non-python developers, too.

[rowanG077]

I'm working on getting a nix environment with cadquery and cq-editor going. Don't have it working yet though.

@adam-urbanczyk A docker image would really help I guess. But it just seems really to extreme to resort to package managers which take over your entire system or having to run an entire second system just to be able to run a piece of software.

[gebner]

@rowanG077 Feel free to look at my nixpkgs fork: https://github.com/gebner/nixpkgs/commits/cadquery CQ-editor doesn't work with the upstream pythonocc, so you might need to change the pythonocc-core sources to https://github.com/CadQuery/pythonocc-core/

[adam-urbanczyk]

@rowanG077 @ilya-epifanov assuming you are on Windows could you try the following way of installing miniconda:

start /wait "" Miniconda3-latest-Windows-x86_64.exe /InstallationType=JustMe /RegisterPython=0 /S /D=%UserProfile%\Miniconda3

In general I use the InstallationType=JustMe and RegisterPython=0 options and I have never experienced issues you are describing.

[tolot27]

It runs fine at my Laptop (Win10 x64), too. The advantage of Conda is that you can create different environments (cqgui in the case of CQ-Editor) with different package versions.

I've created and update.cmd which I use to start CQ-Editor while keeping up-to-date with the sources:

@echo off
git fetch upstream
git checkout master
git merge upstream/master
conda env update -f cqgui_env.yml -n cqgui --prune
conda activate cqgui
python run.py
conda deactivate

[rowanG077]

@gebner Thank you! I got cadquery working in a nix environment now by using part of your commits as a reference. Are you planning on upstreaming? If not I will try to get cadquery and CQ-editor into nixpkgs.

@adam-urbanczyk Isn't it possible to package binary dependencies with pypi? I thought that stuff like python-opencv delivers the opencv binaries with the package itself. Wouldn't this be possible for cadquery as well?

[gebner]

@rowanG077 Not at the moment, feel free to submit a PR to nixpkgs.

[dcowden]

I'm feeling like docker-on-conda is the right answer.

Conda works well when you want full control and full integration into your environment, and for greenfield installation. Docker works well when you want no muss and fuss just works install that disrupts nothing.

So I think the best option is the existing conda based packages, with an additional docker container that has cq already installed (via conda of course)

On Fri, Jun 21, 2019, 6:27 AM Gabriel Ebner notifications@github.com wrote:

@rowanG077 https://github.com/rowanG077 Not at the moment, feel free to submit a PR to nixpkgs.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CadQuery/cadquery/issues/153?email_source=notifications&email_token=AAJ44A74ABS3SX5BEGBSUXLP3SUHLA5CNFSM4HY6ZHU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYIC4NY#issuecomment-504376887, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ44AZSPQ3TAGTNZTYGIMTP3SUHLANCNFSM4HY6ZHUQ .

[adam-urbanczyk]

@rowanG077 @ilya-epifanov any updates on trying the local install?

@rowanG077 yes it is indeed possible to include binary dependencies in a wheel. It is not really practical though, especially for a project with so complex dependencies. If you don't believe me, take a look inside of the wheel that the project you mentioned is providing.

@dcowden agreed. I'll close this issue and open a specific one for docker image. It would be great if someone from the community could pick it up - I don't have the bandwidth and experience.

[adam-urbanczyk]

@jhol I propose to move the discussion here.

[adam-urbanczyk]

I recently found this project which might allow easy conversion of conda to pip packages https://github.com/regro/conda-press I still don't like this approach because it will result in embedding of all the non-python (OCCT, freeimage, ... ) deps in the package. Do we really want to go this way?

[rowanG077]

I have created nix expressions for cadquery and CQ-editor a couple of months back. I still use them. This might also be an alternative instead of pip. Big advantage over conda is that it does not pollute the environment and actually works sanely.

https://github.com/rowanG077/cadquery-nix

[marcus7070]

@rowanG077 Nice to see another nix user! I packaged cq-editor in nixpkgs, it's in 20.03 release. https://github.com/NixOS/nixpkgs/blob/20.03-beta/pkgs/applications/graphics/cq-editor/default.nix

Did you manage to get this working in a non-NixOS environment? I had a go at it but ran into problems connecting the nix installed cq-editor to the Ubuntu installed graphics driver. Here is a discussion on the same problem but for a different package: https://github.com/NixOS/nixpkgs/issues/9415

A solution would be to get the user to run cq-editor through this: https://github.com/guibou/nixGL but it's probably still convoluted enough to turn users away from the nix method and back to conda.

[rowanG077]

@marcus7070 A great! I was planning on getting it into nixpkgs but never got around to it!

I haven't tried it just using nix since I run NixOS. But I'd wager mine has the same problem.

[roipoussiere]

I'm working on CQ-editor flatpak, that could be used for just CadQuery. I keep you informed here: https://github.com/CadQuery/CQ-editor/pull/76#issuecomment-626195409

[vlad0337187]

Used command to install CadQuery, it installed everything into /usr, replaced system python with own and didn't wanted to uninstall it. Decided to reinstall my system.

So I think this issue is actual.

Pip does not really support binary depandancies like oce.

Can't any binary files just be copied using setup.py

I'm working on CQ-editor flatpak

That would be awesome. Use Linux Mint, Fedora, both have preinstalled and configured Flatpak.

[GeorgesPaul]

The advantage of Conda is that you can create different environments (cqgui in the case of CQ-Editor) with different package versions.

The disadvantage is that Anaconda never ever seems to work, whereas in my experience venv (managed by anything but conda), system interpreter en even pipenv work flawlessly every time.

Example (this is me literally trying to get this to work on my system as I type this):

Step 1: conda create -n cadquery

Fatal Python error: init_import_size: Failed to import the site module                                                  
Python runtime state: initialized                                                                                       
Traceback (most recent call last):  File "C:\Python39\lib\site.py", line 73, in <module>                                                                      import os                                                                                                             
File "C:\Python39\lib\os.py", line 29, in <module>                                                                        
from _collections_abc import _check_methods                                                                           
File "C:\Python39\lib\_collections_abc.py", line 12, in <module>                                                         
GenericAlias = type(list[int])                                                                                      
TypeError: 'type' object is not subscriptable          

Fixed the above problem. Let's try again:

Cannot set up a python SDK 
at Python 3.8 (dactyl-python) (C:\Users\xxx\.conda\envs\dactyl-python\python.exe).
The SDK seems invalid.

Wut? Why would it even...!?

uninstall and reinstall anaconda3 a few times to try again

uninstall anaconda3

try miniconda instead

manually set conda.exe env path, because that somehow didn't happen during install

conda activate cadquery

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. 

k? That's probably not supposed to happen uninstalls miniconda

I realize environments are left behind in some directory, removes 6 GB of environments manually

install miniconda again. Installation mentions that it's recommended to start anaconda from start menu to set environment variables

anaconda/miniconda does not appear in start menu

Google how to set miniconda in env path

Can't find conda.exe anywhere, reinstall minconda3 to let installation add miniconda to path, against installation recommendation yet according to documentation recommendation (they contract each other)

Finally figured out what the problem was: I had Python 39 installed, but Conda can only handle up to Python 38 (which is unfortunate, because I would like to use dataclasses which are a Python 39 thing). Deleted all references to Python from environment variables (including Pythonhome/pythonpath), reinstalled minconda, chocolatey refreshenv, closed and reopened terminal and now the conda command works.

On to the next challenge: set up Conda env in Pycharm and see if cadquery will work.

Three hours later: can start coding. Day is over. Have to go to bed.

Disclaimer, this is my personal opinion and I'm probably a n00b, but: I hate anaconda. Messing around with toolchains is my main frustration with coding. This is not the first time random conda crap makes my cry. I've made it work in the past, I've worked with it in the past, but I just can't get it to work now for the above reasons. It's unfortunate that this is not explained in docs that I found and even more unfortunate that Miniconda install doesn't detect that Python is already on the path and might cause problems (and I can't use system python now? I'm confused?). And this is even before all the potential environment and dependency mess. ugggggggggggg. I'm not trying to flame cadquery devs here, I love cadquery itself, it's awesome. My goal is to just give a report of "a random day of a random dude who wants to use cadquery" and the one and only massive struggle that I run into: anaconda.

[adam-urbanczyk]

YMMV but this works in the CI: https://github.com/CadQuery/cadquery/blob/4c77c87eac9ec845a5a17cdd0acad745e938e4b9/appveyor.yml#L17

You might be also interested in using the prepacked version of CQ-editor (and CQ), for now available from here: https://github.com/jmwright/CQ-editor/suites/1584433493/artifacts/28713792

It is not built daily ATM though.

[adam-urbanczyk]

Finally figured out what the problem was: I had Python 39 installed, but Conda can only handle up to Python 38 (which is unfortunate, because I would like to use dataclasses which are a Python 39 thing). Deleted all references to Python from environment variables (including Pythonhome/pythonpath), reinstalled minconda, chocolatey refreshenv, closed and reopened terminal and now the conda command works.

I'm not sure what you mean by "conda cannot handle Python3.9", the following works for me:

conda create -c conda-forge python=3.9 -n py39-demo

Note though that OCP and CQ are not built for Python 3.9 yet.

[GeorgesPaul]

I'm not sure what you mean by "conda cannot handle Python3.9", the following works for me:

conda create -c conda-forge python=3.9 -n py39-demo

Note though that OCP and CQ are not built for Python 3.9 yet.

I'm not sure why the miniconda docs say the following then? I couldn't even get Conda itself to run at all without errors as long as I didn't have Python 3.8 installed on the system (unless conda magically started working due to some other thing a fiddled with).

Or is it that: miniconda needs system Python 38 for itself to run, but can run environments within itself that are Python 39? This was answered below: yes.

[adam-urbanczyk]

That is the base env Python version. You can crate envs with 3.9 (using the above mentioned command).

[GeorgesPaul]

Related question: can CadQuery run on PyPy?

I can't get pypy to run in anaconda, so I'm not sure if it's worth it to keep trying.

[adam-urbanczyk]

No, you cannot run CQ in PyPy.

There are *conda distributions with PyPy: https://github.com/conda-forge/miniforge and you should be able to install pypy (https://anaconda.org/conda-forge/pypy3.7) into a new env in any conda distribution. But let's not make this a conda support line.

[jmwright]

There are two alternatives to using Anaconda now, but they're situational and not a direct replacement for a PyPi package.

1. Install binary development versions of CQ-editor: https://github.com/jmwright/CQ-editor/actions?query=workflow%3Abuild
2. If all you need is to transform the results of a CadQuery script to some output format (STEP, STL and SVG are currently supported), there is cq-cli: https://github.com/CadQuery/cq-cli/releases/tag/v2.1.0

[roipoussiere]

Install binary development versions of CQ-editor

Very interesting! Is it possible to add the compiled binaries in CQ-editor releases artifacts?

Then add download links that points to artifacts of the last version in the readme. The installation will be easier! :)

[jmwright]

I think that @adam-urbanczyk wants to adapt the builds for Azure pipelines first, but that is the general plan.

[Hades32]

I'm using CD-editor for previewing my work but want to use a proper IDE (e.g. VS Code) for development - especially for code completion. Therefore I installed the pip package but now have realized that it's actually the wrong version...

For my specific use-case, having a pip package that doesn't include the binaries would already be pretty helpful!

[rosejn]

I also feel that Conda as a requirement is really hurting cadquery, and I know I'm not alone because as I've been reading up on CQ I find many people complaining about this. Conda is a niche tool hated by many, and the entire python ecosystem is based around pypi. There are many, many python packages deployed via pypi that require C libraries (See numpy, scipy, tensorflow, etc...) so this has got to be possible, and I'm happy to help figure it out. Since the Open Cascade libs are already available as packages for Ubuntu then this might be as trivial as doing an apt install of the open cascade libs and then pip install of cq, which would be python only. When I clone the repo I'm able to directly run python in the base directory and import and use cadquery without a problem. This isn't an exhaustive test though, and I'm not sure what other dependencies there might be. Are the OCE libs the only C library dependencies? We can probably get the pure python release up quickly, and then I'm also 100% positive that the compiled libs can be directly included in the pypi package, because tensorflow does this and it includes binaries not only for the C library but also GPU shaders and more.

CadQuery is a tool I've been hoping for, and I'd love to help get it out to the larger python community. Once people can easily pip install and incorporate into their projects this could really take off.

[jmwright]

@rosejn OCP is our set of bindings to the OCCT CAD kernel, and will have to be handled too.

[roipoussiere]

There is also a Conda dependency with Spyder.

[jmwright]

@roipoussiere I think that will only come into play with CQ-editor. Installing the core CadQuery library via pip shouldn't need it.

[dcowden]

@rosejn I'm no expert on pypy, but wouldnt the right approach to making it just work be to build wheels for the various platforms?

[adam-urbanczyk]

I'm using CD-editor for previewing my work but want to use a proper IDE (e.g. VS Code) for development - especially for code completion. Therefore I installed the pip package but now have realized that it's actually the wrong version...

For my specific use-case, having a pip package that doesn't include the binaries would already be pretty helpful!

@Hades32 you can use git directly with pip: pip install git+https://github.com/CadQuery/cadquery.git

I also feel that Conda as a requirement is really hurting cadquery, and I know I'm not alone because as I've been reading up on CQ I find many people complaining about this. Conda is a niche tool hated by many, and the entire python ecosystem is based around pypi.

@rosejn let's agree to disagree about conda. From where I sit it is not niche. There are people complaining all the time, but somehow I haven't see anyone producing a working wheel yet. It is possible, the question is if it is practical and how much time will we spend on doing it and then supporting users in debugging a zillion of issues specific to different system configurations, incompatible compilation flags of OCCT etc. If you can contribute a build pipeline (preferably azure) for generating wheels for Win, Linux and OSX that work reliably then we'll be more than happy to merge it.

You can also consider building packages for your favorite distro - no blessing from the CQ team is needed to do that.

And BTW I hear that there is a deb package for miniconda nowadays. I haven't tried it myself, I always install to a local dir, never update .bashrc and I never had an issue. Same on Windows.

[rosejn]

OK sure, I'm not here to get into a debate. I'm hoping to be constructive and to help get this project into the mainstream python ecosystem. If you go to scipy, numpy, tensorflow or virtually any other major python library you will find instructions to use pip or related tools like poetry, but nothing about Conda. I'm sure plenty of people use it and it does nice things, so I'm not at all trying to denigrate anyone's toolset of choice. I just know that I have fought with conda in the past, and myself and my team are all setup using poetry across many projects and I don't want to start requiring everyone to use conda.

I'm happy to try and get the ball rolling with OSX and Ubuntu compatible installs, as those are what I've got access to. Hopefully if those are working properly then some inspired windows developer on this thread can help to make it work there as well.

I've never used Azure, but I'm also happy to structure things as a build pipeline if they have some kind of standard setup to drop things into. For now I'll focus on just getting a local dev build working smoothly, and then I'll dig into the cloud automation once we know the key pieces to put in place.

So I've cloned cadquery, OCP, and pywrap. It seems that pywrap is more of a code analyzer that is generating C python bindings, and my assumption is that you've already got this integrated into your build pipeline, correct? It seems like a great goal to have that running 100% automatically on OCCT, and I applaud the effort to make that work. I've done this in the past and for a library as large as OCCT I can imagine that has been a real effort.

Now I'd like to build OCP and CadQuery, but I suspect the build instructions on the README page haven't been updated in a while. pywrap doesn't seem to be an executable for one, and this doesn't seem like a typical cmake setup where you would configure and build from inside of a build directory. Could someone please update the README, or outline the build steps here? Or if there is an azure build pipeline log that you could share which shows the steps maybe that will help.

Your work is greatly appreciated, and I will try to be a net positive rather than take a bunch of your time. I'm an experienced developer with grit so I won't sit around complaining and asking for help once I'm off and running.

[roipoussiere]

If this can help you to achieve a pip module, here is my attempt to create a CadQuery Editor flatpack (hence my confusion about Spyder), and also OpenCascade, where you can find some build steps.

I tried this, then focused on other projects ...so it's not finished. But maybe the build instructions could help you in a way.

[jmwright]

@rosejn There are some build instructions in this yaml file, and there are a few discussions on the Google Group and GitHub on building OCP/Pywrap on FreeBSD, which might include some info that will be useful to you.

@roipoussiere Thanks for the link. I was not aware of your work on the flatpak.

[marcus7070]

Now I'd like to build OCP and CadQuery, but I suspect the build instructions on the README page haven't been updated in a while. pywrap doesn't seem to be an executable for one, and this doesn't seem like a typical cmake setup where you would configure and build from inside of a build directory. Could someone please update the README, or outline the build steps here? Or if there is an azure build pipeline log that you could share which shows the steps maybe that will help.

Azure runs https://github.com/CadQuery/OCP/blob/master/azure-pipelines.yml, which in turn runs https://github.com/CadQuery/OCP/blob/master/build-bindings-job.yml. Here is a recent build log, you can get them through the check mark next to each commit on the OCP page.

If you plan to operate outside of Conda, you'll probably have to build OCCT too. See here for the receipe: https://github.com/conda-forge/occt-feedstock You'll get frequent segfaults if you just use a default build, you need those flags and patches.

Some additional help might come from @greyltc's AUR pkgbuild https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=python-ocp

Are you sure you don't want to try Conda-press first?

[adam-urbanczyk]

OK sure, I'm not here to get into a debate. I'm hoping to be constructive and to help get this project into the mainstream python ecosystem. If you go to scipy, numpy, tensorflow or virtually any other major python library you will find instructions to use pip or related tools like poetry, but nothing about Conda. I'm sure plenty of people use it and it does nice things, so I'm not at all trying to denigrate anyone's toolset of choice. I just know that I have fought with conda in the past, and myself and my team are all setup using poetry across many projects and I don't want to start requiring everyone to use conda.

You don't want to get into a debate and then you continue with it. Please read the official installation instructions of scipy: https://www.scipy.org/install.html#scientific-python-distributions-recommended . Is pip the recommended way of installation? No. Please don't spread misinformation.

Now I'd like to build OCP and CadQuery, but I suspect the build instructions on the README page haven't been updated in a while. pywrap doesn't seem to be an executable for one, and this doesn't seem like a typical cmake setup where you would configure and build from inside of a build directory. Could someone please update the README, or outline the build steps here? Or if there is an azure build pipeline log that you could share which shows the steps maybe that will help.

This is the main repo you need to look at https://github.com/CadQuery/OCP . README is up to date (bottom part is of interest to you), azure build logs can be found behind the azure badge. As mentioned above there have been ppl that did it on AUR, NIX and FreeBSD.

[marcus7070]

There is an interesting output you can get from Conda with the command conda create --dry-run --json -n dummy <package>. Conda will print all the actions it will take to create an environment with package. In particular, there is a link section showing the runtime closure (all the deps required to run package).

Scipy has 30 packages, Tensorflow has 82 and OCP has 93. CadQuery is 119. Just for giggles, CQ-editor has 216.

Cleaned up package lists: scipy.txt tensorflow.txt ocp.txt cadquery.txt cq-editor.txt

If you were a perl or json whiz, I think you could also extract the downloaded size of the runtime closure as well.

[pietakio]

We'd all love to see PyPI-hosted packages for CadQuery. No one sane is opposed to that. Unfortunately, the core issues here are all on PyPI's side: e.g.,

• Distributing large binary payloads as separate downloads.
• API to get dependencies without full download.

Without significant movement from the official PyPI community on those issues (which seems unlikely, because there's no consensus as to at what the high-level solutions to those issues even are), it's unclear whether CadQuery can meaningfully do anything here.

Explain It To Me Like I'm 28

Official Python packaging standards basically prevent sharing large low-level non-Python dependencies like OpenCASCADE (OCCT) and Compute Unified Device Architecture (CUDA) as binary Python wheels that can be safely reused by other Python packages. That's bad. Every Python package requiring those sorts of dependencies has to rebundle those dependencies inside every binary Python wheel they publish. That's even worse. PyPI has strict filesize requirements, because bandwidth funding is understandably scarce. The CDN hosting PyPI downloads currently pays $1.5 million USD out-of-pocket to host those downloads, which is probably unsustainable for obvious reasons.

The full OpenCASCADE distribution alone consumes ~1.5GB in space. Now add pywrap + OCP + CadQuery itself + {whatever-else} to that. Since that math fails to add up on my primitive abacus, it rapidly becomes clear that CadQuery won't be publishing binary PyPI wheels anytime soon — however much everyone wishes it just would.

Can't CadQuery Just Do What Do Other Packages Do?

Different Python packages circumvent those constraints in different ways.

The CUDA-based CuPy package, for example, laboriously publishes one PyPI-hosted binary wheel of CuPy for each supported version of CUDA. Why? Because each version of CUDA is so grotesquely big, the only way CuPy can feasibly support all supported versions of CUDA is to isolate each CUDA version to its own version-specific CuPy wheel (e.g., cupy-cuda112 for CUDA v11.2, cupy-cuda111 for CUDA v11.1). Of course, that's blatantly insane. It makes the pip constraints solver (especially with respect to pip install --upgrade) useless, because pip thinks those are completely different packages. Moreover, it requires CuPy developers to continually go to the PyPI community with their hat in their hands and politely beg for increases in their filesize allotment. Why? Because CUDA ain't gettin' any smaller, folks. At some point in the all-too near dystopian future, PyPI maintainers will be forced to refuse their polite but gluttonous requests for increased bandwidth — at which time the CuPy userbase will be quietly required to use conda to install CuPy. That probably sounds familiar, doesn't it?

Other packages like nltk provide a crazy download() function that you have to manually call after installing a negligibly small PyPI-hosted nltk package shim to actually install the package properly. That's probably even worse than CuPY's kludge, because it breaks idempotency ^{don't forget to remove that nltk.download() call before deploying to production, folks!}, subverts sane packaging standards, and presumably fails to sanely interoperate with pip-based downloads caching. The burden of installation shouldn't be on the end user; it should be on the developer.

There's actually a heated discuss that unsurprisingly went nowhere about this exact topic on both the official Python mailing list and HackerNews. Because PyPI maintainers failed to reach a consensus agreement, the only certainty there is that PyPI (and thus pip) remains hostile to large binary dependencies (and thus Python packages requiring large binary dependencies).

That's us.

So What Should CadQuery Just Do What Do Other Packages Do?

Absolutely nothing. The more pertinent question is: "What should everyone else do?" And the answer to that question is clear: "Drop pip like hot radioactive lead."

pip is great for managing pure-Python packages. pip is okay for managing non-pure-Python packages requiring small binary dependencies. pip is fundamentally unsuitable for managing managing non-pure-Python packages requiring large binary dependencies.

That's not going to change anytime soon... and that's not CadQuery's fault. It is what it is. :shrug:

[Georges-Paul]

That's a really good explanation!

Off-topic rant: Also I would say the solution (to share large binary files within pip and their bandwidth cost problem) to me seems SUPER obvious and relatively simple to implement: the PyPI folks could build a torrent client into pip and download binaries and even pure Python packages that way. At least make the torrent thing an option for large binary files, besides the current way of doing things?

It would still allow them to pay crazy amounts to host (torrent seed) downloads so they remain fast (if they wanted to) + it would allow package distributors (and package consumers) to host/seed binaries and python packages. This would probably greatly decrease the hosting costs for the PyPI folks as well.

Anybody who would like to support Python and pip could simply download or update a torrent magnet link and start seeding. On their server, on their phone or wherever they want.

Aren't these issues basically the reason why the Torrent system was invented?

[fedorkotov]

@as08gf9uw0tjawtegi Torrent means unpredictable bandwidth and availability. "New and shiny" stuff would be shared by a lot of seeders all over the world but older versions of packages that are not used by a lot of people may become unavailable or available at uncomfortably low download speeds. Not to mention the great effort of integrating torrent client with pypi and backward compatibility problems this radical change would bring. In any case if you still believe this is a good idea you should propose it at https://discuss.python.org/

[Georges-Paul]

@as08gf9uw0tjawtegi Torrent means unpredictable bandwidth and availability.

I don't understand this argument. How is a traditional webhost with x amount of bandwidth hosting a Python package more predictable and available than that exact same host seeding a torrent of the same package with the exact same amount of bandwidth?

Do you believe that hosting bandwidth is somehow magically "wasted away" into the torrent network when you seed a torrent vs host a file the traditional way?

[fedorkotov]

Do you believe that hosting bandwidth is somehow magically "wasted away" [..]

No magic involved. Just human nature. People tend to seed only what they use/watch/otherwise utilize now. Few seed something for the greater good. Even fewer do it from always-on server. Because of that some obscure and old packages would be hard or impossible to download. This is what I was talking about.

This is not the place to discuss this initiative. CadQuery is not PyPI. If I'm wrong you probably should pitch this idea to pip maintainers at https://discuss.python.org/

[fedorkotov]

For anyone interested in @as08gf9uw0tjawtegi's torrent-based binary hosting for pip idea. I see they have started a topic at https://discuss.python.org/t/torrent-client-in-pypi-for-package-binary-distribution/9417

[dstufft]

FWIW, it doesn't matter to me whether cadquery hosts on PyPI/supports pip install at all or not. I think every project should support the tooling that works best for them and their users. However I just wanted to clarify/point out a few things as a PyPI maintainer/admin!

Easiest/happiest thing first.

• API to get dependencies without full download.

This is coming, we've had a rough idea of what to do for awhile now, and we were just missing someone with the free time to go through and write it up as a proper PEP. That happened and PEP 643 is now accepted and is currently being implemented.

• Distributing large binary payloads as separate downloads.

This is still a problem. In the linked thread under "heated discussion" (which I don't think was actually heated! but that's neither here nor there) Dustin (one of the PyPI maintainers/admins with me) laid out the problem with several possible solutions. We don't have a solution to this, but I'm pretty sure that everyone involved wants to solve this problem. Right now it's stuck in the "discuss possible ideas" phase, largely because nobody has stepped up to write, submit, and champion a PEP for a possible solution (not that that PEP would be guaranteed to be accepted, but that's how those projects make decisions).

The Venn diagram of people who have the "large binary" problem, people who care/want to work on pip/PyPI, and people who have the time to work on the above is pretty small, so it'll likely languish until someone gets more time, it bubbles up to be a much larger problem, or someone external champions an idea that gets successfully accepted.

[jchidley]

I have CQ-editor working on my "macOS Big Sur" "MacBook Air M1" (Apple Silicon ARM based chip). I also use Windows 10 devices. I really can't remember how I installed CQ-editor (possibly the binary download?).

CQ-editor (and Jupyter-CadQuery) are all very nice but I'd like use a proper IDE (in my case Visual Studio) in order to better use and understand Cadquery. This has proved extremely difficult and hence, in my research, I found this thread. Sadly, even after reading everything above, I still don't have a working solution.

To increase adoption of cadquery, a key requirement is easy installation of it. That means that either installation should be a single operation or that there should be concise and clear documentation to follow (actually, there should always be such documentation).

I avoid using conda because I find it buggy, difficult to use and because it causes conflicts on my systems.

Should I ever manage to get this to work, I'll be documenting it.