setup.py error: can't create or remove files in install directory (Windows & Ubuntu)

[BurtHarris]

Sorry, the root cause is clearly that I'm a python newbie. I'm running on Windows 10 with python 3.9 installed from the Microsoft Store.

Python 3.9.4 (tags/v3.9.4:1f2e308, Apr 4 2021, 13:27:16) [MSC v.1928 64 bit (AMD64)] on win32

Following the readme, I've cloned from github and run setup:

PS C:\try\pcbflow> python setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.9_3.9.1264.0_x64__qbz5n2kfra8p0\\Lib\\site-packages\\test-easy-install-18976.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1264.0_x64__qbz5n2kfra8p0\Lib\site-packages\  

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://setuptools.readthedocs.io/en/latest/easy_install.html

Please make the appropriate changes for your system and try again.

[BurtHarris]

I'm guessing that this could be addressed by running in administrator mode, but I try to avoid doing that, especially with software I don't know much about.

Is there some way to install needed packages into some other folder?

Would a python virtual environment help me?

[BurtHarris]

I'm looking at a package manager called pipenv, which seems natural to me (coming from node.js); so I'm going to give a try it to set up a virtual environment.

Thoughts?

P.S. I got a little into using pipenv for setup but got stuck on an error ending in:

OSError: no library called "cairo" was found
cannot load library 'C:\Program Files (x86)\FAHClient\libcairo-2.dll': error 0xc1
cannot load library 'libcairo.so.2': error 0x7e
cannot load library 'libcairo.2.dylib': error 0x7e
cannot load library 'libcairo-2.dll': error 0x7e

[michaelgale]

Sorry I can't help you with this one since I'm not a Windows user--I can't really explain the messages you're getting.

Something that might help is using "virtual python environments". I happen to use Miniconda, but there's also anaconda, mamba, and a few others. Virtual environments are typically a good idea in any case since you can have dedicated lightweight python environments for different tasks. They are also useful for resolving package dependencies such as the error message above for "cairo".

[BurtHarris]

Thank you. I've got a lot of Windows background and some time to invest if you are interested in collaboration.

My first thought is to do Windows-based CI builds using appveyor.com, as described in the python docs. Appveyor is a lot like Travis.ci, but more mature in its Windows support. I just started into exploring that before reading your reply:

github branch: https://github.com/BurtHarris/pcbflow/tree/win32 appveyor status: https://ci.appveyor.com/project/BurtHarris/pcbflow

I'll trim down the environments list, for now, to make the builds run faster; what python version would you recommend?

The recommended Appveyor instructions for python seem to use a packaging framework called wheels. I gather you are using something older and/or homegrown. Do you know anything about wheels? Does it make sense to use for pcbflow?

[michaelgale]

In terms of python version, any version 3.6 or higher will work. If you look at this repo, you'll see that the Travis CI configuration already tests 3.6, 3.7, 3.8. Furthermore, you'll see that the Travis CI config uses miniconda to setup a virtual environment based on the environment.yml file in the repo. This could give you clues on how to use a miniconda virtual env if you choose go that way.

My python setup.py script is a standard issue setuptools script file that actually builds "wheels" distributions. When you run the setup.py script it makes a build and dist folder with the packaged wheels files. Of course it can only do this for the platform that it is running on. In my case, it is macOS, when it runs on Travis, it is linux. A Windows virtual env should work the same way--and it does, since I use this same setup.py script on all my python packages and I know that folks who use my packages run Windows, macOS, and linux. I didn't make the setup.py script, it is a template which I replace with relevant info such as the package name, version, etc.

If I were you, I would setup a working virtual python environment. Then I would explore the use of package installation tools such as pip and setuptools. It can be a bit intimidating and unfamiliar at first, but once you get the hang of it, python virtual environments are really quite useful and installing packages and dependancies is actually pretty reliable and fuss free.

[BurtHarris]

OK, thanks. Understood.

[BurtHarris]

I tried this on an Ubuntu virtual machine, got a very similar message.

burt@Gladiator:~/try$ git clone https://github.com/michaelgale/pcbflow.git
Cloning into 'pcbflow'...
remote: Enumerating objects: 641, done.
remote: Counting objects: 100% (641/641), done.
remote: Compressing objects: 100% (451/451), done.
remote: Total 641 (delta 439), reused 386 (delta 184), pack-reused 0
Receiving objects: 100% (641/641), 4.83 MiB | 2.85 MiB/s, done.
Resolving deltas: 100% (439/439), done.
burt@Gladiator:~/try$ cd pcbflow/
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python3.8/dist-packages/test-easy-install-143.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.8/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://setuptools.readthedocs.io/en/latest/easy_install.html

Please make the appropriate changes for your system and try again.

burt@Gladiator:~/try/pcbflow$

[BurtHarris]

FYI, I finally got virtual environment creation and activation under Ubuntu to work, and then running python3 setup.py install seemed to complete successfully.

[BurtHarris]

@michaelgale, I posted about this on the python discourse server. The responses that came back, including some from core python developers, seem to say that running a setup.py directly is not recommended, that packages should be installed through pip. See details at:

https://discuss.python.org/t/should-the-python-windows-store-package-have-a-default-virtual-environment/8288/4

[BurtHarris]

@michaelgale, In the readme file's install instructions, you have $ python setup.py install. Is there some reason you aren't using the command $ pip install . instead?

That seemed to be the consensus in the abovementioned discussion. It seems to work on every version of Python on Windows I tried.

[alphakay9]

"python setup.py install --user" worked for me

[qurbat]

@michaelgale @BurtHarris can this be closed with the answer @alphakay9 provided?

[michaelgale]

@qurbat Sure. This is obviously a stale thread and installation with setup.py seems to work on most platforms.