search

conda / constructor

tool for creating installers from conda packages
https://conda.github.io/constructor/
Other
456 stars 167 forks source link

Struggling with permissions issues for the AllUsers settings on Windows #828

Open moorepants opened 1 month ago

moorepants commented 1 month ago

Checklist

What happened?

We build a constructor Windows installer annually and install for AllUsers in silent mode. This year many/most of the installed scripts in <install-directory>/Scripts/ have permissions that prevent execution. For example, opening the Anaconda prompt and typing Jupyter lab gives Error executing Jupyter command 'lab': [WinError 5] Access is denied. Similar errors are produced when trying to launch lab from the Anaconda Navigator. Running spyder from the prompt or Navigator works. We tried building the new installer with the constructor version (3.4.5) we used last year but still get the permission errors. We have many details of our trial and error in this issue: https://gitlab.ewi.tudelft.nl/bhmgerritsen/anaconda-dee-config/-/issues/64 but I'll summarize below.

Constructor setup:

If we download the latest official Anaconda windows installer and run it in silent mode the permissions are correct. But our installer leaves many files with restricted permissions.

Changes from last year:

We have run out of ideas as to what causes the permissions errors. Do you have any suggestions for what we may be doing wrong?

Conda Info

NA

Conda Config

NA

Conda list

NA

Additional Context

No response

marcoesters commented 1 month ago

I downloaded your 2024.01 and ran the GUI installation - everything looks good on my end. jupyter lab launches from the prompt without problems. I ran it on a Windows 11 machine where I'm the only user.

The fact that this also breaks with the old constructor makes me think that there is a specific interaction with your system. Does it maybe restrict what unsigned installers and/or binaries can do? The Anaconda and, since recently, the binaries on Windows are all signed by Anaconda.

You could also check if installing the package from defaults instead of conda-forge helps.

moorepants commented 1 month ago

Thanks @marcoesters, good to know that it worked for you. We can look more closely at things in the environment we are installing to. I'll ask out IT if we need the signing and we're already working on checking the installs from defaults. I'll report back on our success/failure.

moorepants commented 1 month ago

@marcoesters did you do a "JustMe" installation? Because that does seem to work for us. It is the "AllUsers" option that has the borked permissions.

moorepants commented 1 month ago

One other important note I forgot from above, is that the installer we built last year works just fine with and AllUsers install on the computers this year. So that is odd.

subsense33 commented 1 month ago

To reproduce the error, it is important to start the installer as the local SYSTEM account, not just as a local administrator. Our software deployment mechanism also installs software using the local SYSTEM user.

You can use Microsoft psexec.exe to start an elevated command prompt as the SYSTEM user.

  1. Open an elevated Command Prompt.
  2. Run the following command: psexec.exe -s -i cmd
  3. A new Command Prompt window will open. To verify your current user, type whoami.
moorepants commented 1 month ago

To reproduce the error, it is important to start the installer as the local SYSTEM account

Does that mean you have reproduced the error? This is exactly how we are installing using the SYSTEM account.

Oddly, the installer we build last year installs with correct permissions with the SYSTEM account but the new installer (built with either constructor 3.4.5 or the latest version) has incorrect permissions with the SYSTEM account.

moorepants commented 1 month ago

Ah, I didn't realize @subsense33 was my colleague here at TU Delft. He's just explaining what we've done. I assumed @subsense33 was another person commenting on this.

moorepants commented 1 month ago

I just installed the v2024.01 release from https://gitlab.ewi.tudelft.nl/bhmgerritsen/anaconda-dee-config/-/releases on a regular Windows PC. I selected the "All Users" option and to "Register Python". I elevated admin rights to run the installer and then afterwards checked running spyder, jupyter, python console, ipython both from command line and anaconda navigator app and everything worked correctly with no permissions issues. This is presumably what @marcoesters did.

So, this points towards some kind of issue installing with the local SYSTEM account. (except that last year's installer has no issue installing with the SYSTEM account)

marcoesters commented 1 month ago

Thanks @subsense33 for these instructions! I was able to reproduce the problem now, but I suspect this is not related to constructor or your installer, but to jupyterlab. Here is what I did to come to that conclusion:

My hypothesis is that jupyterlab tries to write files into its own install location, which is write-protected. This would also explain why rebuilding the installer with an older constructor version doesn't work either. Your installer from last year (I assume) contains an older version of jupyterlab, which may not exhibit this behavior.

marcoesters commented 1 month ago

Of course, executing jupyterlab from the new environment means that I have full permissions to that executable as well. However, the fact that I can execute all other binaries in the base environment tells me that jupyter tries to do some kind of write operation into a location that it's not allowed to do.

marcoesters commented 1 month ago

I also manually checked the permissions inside the Scripts folder and they are what I expect: as a user, I only have read and execute permissions. The WinError is also a python error, so it reaches and executes the python interpreter somehow. If this was a permission error of your directory, I'd expect the error to be emitted by Windows, not python.

moorepants commented 1 month ago

Thanks for the insight. The permission error also occurs when trying to run ipython from the anaconda prompt.

We could install an older version of IPython and Jupyter as a stopgap.

moorepants commented 1 month ago

Here is an issue on the IPython tracker that implies that IPython tries to create something in the user's home directory:

https://github.com/ipython/ipython/issues/12079

It is from 2020 though, so that doesn't align with our last year's installer working.

moorepants commented 1 month ago

Searching for "jupyter" and "access is denied" gives lots of hits, like: https://stackoverflow.com/questions/61747187/jupyter-lab-in-anaconda-not-working-error-executing-jupyter-command-lab-win

moorepants commented 1 month ago

It's worth noting that we see this: image

indicating that we don't even have read and execute permissions for all the binaries.

marcoesters commented 1 month ago

You are right, alembic and other binaries don't have the correct permissions. jupyter does though.

For AllUsers installations, the permissions are supposed to be fixed here: https://github.com/conda/constructor/blob/main/constructor/nsis/main.nsi.tmpl#L1240-L1253

You can monitor the installation and see that you will lose read access to the entire directory during the installation, but get it back shortly after. Maybe the permissions changes are not performed recursively? I will have to investigate what those commands do exactly, but that change is also two years old, so this cannot be the only factor.

Alembic doesn't have read access throughout the entire installation process, even before the permissions are fixed in the lines above.

You may be running into a few things at once here:

moorepants commented 1 month ago

I will have to think about whether this comes with security risks, even though read and execute privileges should be relatively safe.

Our current work around is to give everything in <install-dir>/Scripts read/exec permissions in a post install script.

subsense33 commented 1 month ago

I will have to think about whether this comes with security risks, even though read and execute privileges should be relatively safe.

Our current work around is to give everything in <install-dir>/Scripts read/exec permissions in a post install script.

Yesterday I changed the work-around script to reset the owner on <install-dir>. Not just <install-dir>/Scripts. Does no harm, just to be sure everything is set correctly. The script resets the owner of the entire tree to the local group "Administrators", that is how it's supposed to be. (Like the Anaconda deployment of past years).

Normal users have only read and execute permissions, that is safe. So something weird is happening with the "owner" of certain files. I use the Alembic binary just to check if the problem is there, but more files are affected.

marcoesters commented 1 month ago

While that workaround probably works, the ownership of the files is not different from the Anaconda installer. I am using jupyter-kernel.exe as an example here. File permissions are listed using Get-Acl | fl.

Anaconda installer, installed as SYSTEM:

Path   : Microsoft.PowerShell.Core\FileSystem::C:\ProgramData\anaconda3\Scripts\jupyter-kernel.exe
Owner  : BUILTIN\Administrators
Group  : NT AUTHORITY\SYSTEM
Access : BUILTIN\Users Allow  ReadAndExecute, Synchronize
         NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Administrators Allow  FullControl
Audit  :
Sddl   : O:BAG:SYD:AI(A;ID;0x1200a9;;;BU)(A;ID;FA;;;SY)(A;ID;FA;;;BA)

TU Delft installer, installed as SYSTEM:

Path   : Microsoft.PowerShell.Core\FileSystem::C:\ProgramData\tudelft-anaconda\Scripts\jupyter-kernel.exe
Owner  : BUILTIN\Administrators
Group  : NT AUTHORITY\SYSTEM
Access : NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Administrators Allow  ReadAndExecute, Synchronize
Audit  :
Sddl   : O:BAG:SYD:PAI(A;;FA;;;SY)(A;;0x1200a9;;;BA)

So, the owner is the same, but for some reason, access isn't granted the same way. I'm wondering if this is because of differences between the two build systems (Anaconda vs. conda-forge).

I'd be really curious to see if the problem persists if you build the installer using only defaults packages.

moorepants commented 1 month ago

This was my attempt at building the new installer the same way we did last year (defaults as priority w/ conda forge packages on top):

https://github.com/moorepants/anaconda-dee-config/actions/runs/10185441100

We have the same permissions issue with it.

I can build an installer without any packages from conda forge, but I just have to go through an make a new construct.yaml that eliminates all packages from conda forge. Last year's approach included the anaconda package (as does the one I linked just above). Are you suggesting to build an installer only from defaults package but without using the anaconda package or to include it?

Note that I switched to the "conda forge only" approach because it has long been recommend to not mix conda forge packages with defaults packages by conda forge. Our users need packages only available on conda forge, so I went the "pure" conda forge route to ensure binary compatibility.

marcoesters commented 1 month ago

I suggest to not use conda-forge as a channel for this test, yes. I'd also reduce the number of packages since we know which packages are problematic: the packages that install alembic or jupyter-kernel. That just makes debugging faster.

If that results in the same problem, I'm out of ideas. The workflow looks okay. The only difference I see is that you use PowerShell instead of bash (as Miniforge and Anaconda do), but that can't be the reason, can it?

moorepants commented 1 month ago

I built an installer without conda-forge packages in the base environment. You can download here:

https://github.com/moorepants/anaconda-dee-config/actions/runs/10242877472

Note that my install action fails with:

Error: Unable to locate executable file: C:\Users\runneradmin\miniconda3\condabin\conda.bat. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also verify the file has a valid extension for an executable file.

I'm not sure why. I did not include the anaconda package.

moorepants commented 1 month ago

PowerShell instead of bash (as Miniforge and Anaconda do), but that can't be the reason, can it?

I can test a build using bash to call the constructor related commands.

marcoesters commented 1 month ago

I built an installer without conda-forge packages in the base environment. You can download here:

https://github.com/moorepants/anaconda-dee-config/actions/runs/10242877472

Note that my install action fails with:

Error: Unable to locate executable file: C:\Users\runneradmin\miniconda3\condabin\conda.bat. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also verify the file has a valid extension for an executable file.

I'm not sure why. I did not include the anaconda package.

Probably because the installation has failed. I'd run the GUI with the artifact and/or create an environment with your specs to see if there is a package conflict.