Update Windows installer to not require administrator privileges

smalers commented 2 years ago

As of TSTool 14.0.3, the Windows installer requires administrative privileges. This is mainly due to how the NSIS installer modifies the registry for the uninstall tool. The software itself installs into C:\CDSS\TSTool-* folder, which does not require special privileges. Windows also now supports user application files in the AppData folder under a user account, for example to allow a user to install Python modules. It should be possible to update the TSTool installer to not require administrative privileges, but this needs to be researched and prototyped.

smalers commented 1 year ago

This might be relevant.

I'd need to try it and see how it works. It might also be a good time to revisit how the uninstaller works. I've noticed that sometimes not all the files are removed.

smalers commented 1 year ago

For the CDSS development environment setup, the recommendation suggests modifying the C:\Program Files (x86)\NSIS\Contrib\zip2exe\Base.nsh file and adding the following:

RequestExecutionLevel user

I made the change and ran the process to create the installer. I'll have someone from the State try to install to see if it can be done without administrative privileges. Here is the link to the unpublished 14.7.0 installer:

https://opencdss.state.co.us/tstool/14.6.0/software/TSTool_CDSS_14.6.0_Setup.exe

smalers commented 1 year ago

The NSIS installer tutorial has an example for removing shortcut menus: https://nsis.sourceforge.io/Simple_tutorials

This is the critical step for an uninstaller. I think the existing installer relied more on some built-in Microsoft uninstall process that required setting a registry setting.

Because the installer only creates one menu shortcut, I don't think we need to set the registry variable. I recommend testing this and seeing if the admin privilege requirement for the installer can be removed.

smalers commented 1 year ago

After updating the installer to not write registry settings, administrator privileges are still required to install. I'm doing a deeper dive with research shown below and recommendations at the end.

Installing TSTool in a shared location or by user

TSTool is currently installed in C:\CDSS\TSTool-Version which is considered installing in the system root folder. Hence, installing in C:\CDSS\... requires permissions similar to installing in C:\Program Files\.... An important consideration is whether multiple users will use a computer and want to run the same software. Installing the software per user will limit multiple users from using the same software (hence why software is installed in C:\Program Files) and will require more disk space.

Some software installers (like Python) ask "Do you want to install the software so multiple users can share or install only for the current user?"

The current configuration allows the software to be run by any user on the computer.

It is likely that many TSTool users are professionals and do not share their computers. However, there may be cases such as traveling laptops, computers shared by part-time staff and interns, etc., where shared software makes sense.

Using the C:\CDSS folder for other software and data probably makes some sense. For example, system processes like SQL Server and HydroBase installed in a shared location is used by system processes. Sharing large data files on a computer probably also makes sense. People can install data files in their own folders if they want.

ChatGPT Response about "Requested operation requires elevation"

The "requested operation requires elevation" error message occurs when the installer needs to modify system files or settings that require administrative privileges. To avoid this, you can make some changes to your NSIS script to make sure that it does not require administrator privileges.

Here are some steps you can follow to create an NSIS installer that does not require administrator privileges:

Change the installation directory to a user-specific location: By default, NSIS will install the application in the "Program Files" folder, which requires administrator privileges. You can change the installation directory to a user-specific location such as the AppData folder by using the $APPDATA constant. This can be done by adding the following line to your NSIS script:

InstallDir "$APPDATA\YourApplicationName"

Avoid writing to system folders: If your application needs to write data to the system folders, it will require administrative privileges. You can avoid this by writing data to the user-specific folders instead. For example, you can use the $APPDATAconstant to write data to the AppData folder.
Remove any unnecessary registry entries: If your application needs to create registry entries, make sure they are not in the HKEY_LOCAL_MACHINE key, which requires administrative privileges. Instead, create the registry entries in the HKEY_CURRENT_USER key, which does not require administrative privileges.
Use the RequestExecutionLevel directive: By default, NSIS runs the installer with administrative privileges. You can change this behavior by adding the following line to your NSIS script:

RequestExecutionLevel user

By following these steps, you should be able to create an NSIS installer that does not require administrator privileges. However, keep in mind that some applications may require administrative privileges for specific operations, in which case you may need to prompt the user to elevate privileges.

ChatGPT output about root folder

Yes, administrative privileges are required to create files in the C:\ root folder in Windows. The C:\ root folder is a system folder that contains important system files and settings, and it is typically protected to prevent unauthorized modification or deletion.

By default, standard users do not have write permissions to the C:\ root folder. Attempting to create a file or modify a file in the C:\ root folder without administrative privileges will result in an error message that says "Access is denied."

If your application requires files to be created in the C:\ root folder, it will need to be run with administrative privileges. However, it is generally not recommended to create files in the C:\ root folder, as it can interfere with the proper functioning of the operating system and other applications. It is generally better to create files in a user-specific location, such as the user's home directory or the AppData folder, which do not require administrative privileges.

Installation folder research

I remember now why I have not sorted this out previously: there is not a clear recommendation on where software should be installed. Here is some research. The main things to consider to install software without administrative privileges are:

Is it a good idea to do it?
Where does the software get installed?
How are Start menu shortcuts handled?
How will the uninstaller work?

Is it a good idea to not install as administrator?

There is a lot of information, including the above aggregated by ChatGPT, that describes how to install software without administrative privileges (and avoid C:\Program Files). An examination of my AppData folder shows files for many programs from many software providers. Some are just user data files (profiles, etc.) and others are software executables. So, it is generally accepted.

An analog comparison is that TSTool has the option of being installed in a user's files on Linux (for a user-specific installation) and it works fine.

Where does the software get installed?

Start menus are added for each user. They use Windows shortcuts (links) from the menu configuration to the executable to run. Right-clicking on a menu and viewing its More / Open file location shows where the menu link is stored (and that link points to an executable).

Visual Studio Code (a popular Microsoft development tool):
Menu link: C:\Users\steve\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Visual Studio Code\Visual Studio Code
Link target:  "C:\Users\steve\AppData\Local\Programs\Microsoft VS Code\Code.exe"

TSTool 14.7.0 (before changes to remove administrator privileges requirement):
Menu link: C:\ProgramData\Microsoft\Windows\Start Menu\Programs\CDSS\TSTool-14.7.0
Link target: C:\CDSS\TSTool-14.7.0\bin\TSTool.exe

Notepad++ (a popular text file editor):
Menu link:  C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Notepad++
Link target:  "C:\Program Files\Notepad++\notepad++.exe"

GIMP (popular image capture and manipulation program):
Menu link: C:\ProgramData\Microsoft\Windows\Start Menu\Programs\GIMP 2.10.32
Link target:  "C:\Program Files\GIMP 2\bin\gimp-2.10.exe"

QGIS Desktop 3.22.16:
Menu link: C:\ProgramData\Microsoft\Windows\Start Menu\Programs\QGIS 3.22.16\QGIS Desktop 3.22.16
Link target: "C:\Program Files\QGIS 3.22.16\bin\qgis-ltr-bin.exe"

Python 3.10:
Menu link: C:\Users\steve\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Python 3.10\Python 3.10
Link target:  C:\Users\steve\AppData\Local\Programs\Python\Python310\python.exe

PyCharm Community Edition 2022.2.3 (a popular IDE for software development):
Menu link:  C:\ProgramData\Microsoft\Windows\Start Menu\Programs\JetBrains\PyCharm Community Edition 2022.2.3
Link target: "C:\Program Files\JetBrains\PyCharm Community Edition 2022.2.3\bin\pycharm64.exe"

The AppData folder can contain subfolders. ChatGPT says:

"It is important to note that the exact use of each folder can vary depending on the application and the specific requirements of the data being stored. However, as a general rule, the Local folder should be used for data that is specific to the current user and computer, the Roaming folder should be used for data that needs to be synced across devices, and the LocalLow folder should be used for low-integrity data."

The use of "data" above apparently also includes software files. Another answer from ChatGPT:

"Within the AppData\Local folder, it is common to create a subfolder for each application to store its data and configuration files. The recommended name for the subfolder is usually the name of the application or the name of the company that created the application. For example, if you were creating an application called "MyProgram" and your company was called "MyCompany," you might create a subfolder at: %USERPROFILE%\AppData\Local\MyCompany\MyProgram".

ChatGPT says the following about folder to use for menu links:

"On Windows, menu links are typically saved in the Roaming folder while the actual data used by the application is saved in the Local folder. The reason for this is that menu links are considered to be part of the user's configuration or customization settings, and therefore should be synced across devices if the user has enabled Windows settings syncing."

The following are observations:

Of the above, Visual Studio Code and Python store program files in a user's AppData location, including start menu files and the software. This is the most user-centric.
Visual Studio Code installs one version with updating occurring to that version.
QGIS allows multiple versions to be installed, similar to TSTool, and by default installs software in C:\Program Files and user profiles in AppData.
Python allows multiple versions to be installed, similar to TSTool and installs software files in AppData. Note that it installs into Local\Programs\Python\, which uses the extra Programs folder. Most other software uses Company\Software convention without Programs.
Pycharm allows multiple versions to be installed but for links does not a folder for the link. Note that PyCharm uses an inconsistent start menu link location (not in Roaming, etc.).

How will the uninstaller work?

TSTool uses the NSIS installer software, which has built-in features for adding and removing from the user's start menu, and removing files. It is also possible to use user registry variables. I need to experiment whether it will work as expected. There have been cases in the past where uninstalling removed most but not all artifacts, perhaps because the uninstaller was not complete. Because the software installation is confined to a main folder and the menu, uninstallation should be straightforward.

Some programs have Start menu options with "uninstall". Using this with QGIS opens the Windows uninstall tool and you have to reselect QGIS to uninstall.

TSTool used to show an uninstall choice in the Start menu but that led to a lot of menu clutter. Some things to consider:

It would be possible to put each TSTool version in a menu, like QGIS, and then a submenu would run a specific version.
Or, rely on the Windows remove software tool (with user registry settings?).
Or, have a menu in TSTool to uninstall and then deal with the sequencing of shutting down the software so it can uninstall. Or, only allow uninstalling old versions this way.
Rely on documentation and optionally popup information to explain how to uninstall by running the uninstall program.

Because TSTool now stores some user files in a user's home folder in .tstool folder, uninstalling the software is independent of uninstalling other TSTool files. This can be cleaned up more in the future, perhaps with a TSTool menu to help.

Recommendations

Based on the above, I recommend:

Shared and user installations - A decision needs to be made about whether to allow the software to be installed in C:\CDSS to share between multiple users on the computer, or install in user files per user. Both can be supported but will require changing the installer to prompt for an answer to the question. Documentation will also need to be updated accordingly. Although the installer code will be a bit more complex, I think it may be a good idea to support shared installation (same as current) and user installation (new configuration). I have to make additional changes anyhow and the current approach can be retained as an option. This will also allow the installer code to cleanly branch and show the comparison. It would also provide a good template to follow for other software installers like StateDMI. There may need to be some changes to TSTool to detect where it is installed, but this is typically handled dynamically and may be OK without additional changes.
To enable user installation without administrator permissions:
1. Installing the software in C:\Users\user\AppData\Local\CDSS\TSTool\TSTool-Version\TSTool software files.... In this case, CDSS is the "company". Using a TSTool folder is appropriate because multiple versions of TSTool can be installed at the same time. QGIS and Python do something similar.
2. Save the menu link to something C:\Users\user\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\CDSS\TSTool\TSTool-Version\TSTool-Version (link). Using Roaming (rather than Local) seems to be the precedent even if the software won't be used across devices (see the Visual Studio Code and Python examples). The CDSS/TSTool folders may be unnecessary (other software does not seem to use company) but using will keep consistent with the software installation folder pattern.
3. Test allowing the installer to use registry settings to save the installation location but use current user registry properties, which don't require administrator privileges. This should help the Windows installer tool know where to find uninstall programs. Worst case, don't use user registry settings and instead use TSTool menu and documentation to explain how to uninstall.
4. Use the NSIS RequestExecutionLevel user directive.
In any case - Continue to use the .tstool folder for user's TSTool configuration files. Although AppData could be used, there is enough inconsistency and possible confusion that it does not make sense to change. Also, .tstool is used on Windows and Linux, similar to other programs like AWS command line (.aws/) and Visual Studio code (.vscode/).

I am going to get confirmation from the State of CO before doing any more changes.

smalers commented 1 year ago

Brian Macpherson provided some insight via email:

Just went through an interesting experience installing a new version of RiverWare. Similar to TSTool, you can have multiple versions installed on your machine.

First off, you can see that TSTool has the windows security shield logo in the bottom right of its icon, meaning that right off the bat it asks for admin login.

When I open RiverWare installer, you get to this:

where the default is to install to ...\AppData\..., but the convention seems to have been established that it gets installed in C:\CADSWES\RiverWARE 9.0.4\...

I just tried it and it let me install to C:\CADSWES\, which is odd since it seems analogous to TSTool wanting to install to C:\CDSS\... somehow CADSWES pulled it off.

As it says, if you try to install to C:\Program Files\... it requires admin again (you can see the shield logo indicating that)

so I think in theory you should be able to keep the C:\CDSS\ directory with no admin privilege. Would you like to be able to reach out to their programmers to ask them about it? I could probably connect you.

smalers commented 1 year ago

It is interesting that RiverWare seems to be handled similar to TSTool.

Ability for normal user to write to root folder

I tested whether I could recreate a folder in C:\ on a computer where my account is not an administrator and indeed I could. Maybe some systems don't allow but we can try using it without administrator privileges and see how it goes. If some people have trouble, they can probably run as administrator or pick another installation folder.

Security Icon

Brian's catch on the security icon is useful because that can be checked without trying to run the installer. After some searching I found that Microsoft provides the mt.exe software to manipulate executable files and the <requestExecutionLevel> property is key. I did not install the software but used the vim -b text editor command to edit an installer in binary mode. It shows the following, which includes requireAdministrator:

 manifestVersion="1.0"><assemblyIdentity version="1.0.0.0" processorArchitecture="*"
name="Nullsoft.NSIS.exehead" type="win32"/><description>Nullsoft Install System v3.08</description>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"><security>
<requestedPrivileges><requestedExecutionLevel level="requireAdministrator" uiAccess="false"/></requestedPrivileges></security></trustInfo>

This is controlled by the NSIS RequestExecutionLevel command. I searched the NSIS installer files used by TSTool and did not find. However, the TSTool installer code includes files from the NSIS installed software and I found multiple cases where it is set. I updated the externals/CDSS/installer/cdss.nsh file to set RequestExecutionLevel user after !include statements and recreated the installer. The security decorator is not shown on the program executable icon and I can install without administrator privileges on the normal user account.

Registry settings for uninstall

Previously, the installer set HKLM registry settings, which are only editable by someone with administrator privileges. For a normal user, the HKCU registry settings should be set. I updated the install builder and now the values are similar to the following. I also found a bug where the uninstaller was not completely removing the registry keys, which resulted in the software remaining in the Apps settings listing software.

The uninstaller removes the files from the C:\CDSS\TSTool-14.8.0 folder and the start menu.

Status

I think this is good enough for now. Let's see how the new 14.8.0 installer works for various computer environments.

OpenCDSS / cdss-app-tstool-main