microsoft / azuredatastudio

Azure Data Studio is a data management and development tool with connectivity to popular cloud and on-premises databases. Azure Data Studio supports Windows, macOS, and Linux, with immediate capability to connect to Azure SQL and SQL Server. Browse the extension library for more database support options including MySQL, PostgreSQL, and MongoDB.
https://learn.microsoft.com/sql/azure-data-studio
MIT License
7.56k stars 901 forks source link

Provide a documented method to install Azure Data Studio and dependancies in Secure Environments #13424

Open sqllensman opened 3 years ago

sqllensman commented 3 years ago

Is your feature request related to a problem? Please describe. The current experience of installing Azure Data Studio on systems that are either behind Proxy servers or not connected to Internet is extremely time-consuming and frustrating. When trying to install or use Azure Data Studio on machines that are not left wide open to Internet the following issues always occur

  1. Automatic update of Azure Data Studio never works. Throws a meaningless error. This problem can be worked around by downloading the latest installation file
  2. Installation or update of all of the secondary dependancies has the exact same issues. The notebook "feature" requires a Python installation and the Offline installation of this component is even more hopeless. There is now clear indication of (a) Which version of Python are supported (b) Where this should be downloaded from (c) Which additional pip files are required (and when these change) (d) Where pip files can be downloaded from (e) How pip files are installed (f) The fact that they are changing. Installing Ver 1.2.4 had a hidden upgrade of requiring an upgrade of the PowerShell pip file.
  3. All of the VSIX files also need to be manually downloaded and installed.

Describe the solution or feature you'd like A clearly documented method to download and install ALL of required components and not just the main executable. The design of Azure Data Studio is such that almost all functionality requires additional components and these should be clearly listed and available to install New releases should clearly document when they are dependant on upgrades to other components.

Describe alternatives you've considered The only alternative outside of spending hours trying to resolve the poor offline installation experience is to simply abandon use entirely. Azure Data Studio without extensions and Python installed is basically unusable.

Additional context The same program that cannot install any of its own required extensions can successfully install the latest version of Powershell. After having to manually install Ver 1.2.4 I was able to update Powershell to V7.1 .

SQLDBAWithABeard commented 3 years ago

I have been trying to get this to work using a windows 10 hyper-v machine, the dev quick create version.

I failed to get the notebook experience working. I followed these steps

Download ADS 1.25.1 user from Download and install Azure Data Studio - Azure Data Studio | Microsoft Docs

Download Python 3.9.1 from Download Python | Python.org

Download notebook zip 6.2 from Releases · jupyter/notebook (github.com)

Download node.js from Download | Node.js

Create dir Create constraints.txt

Add backcall==0.1.0 bleach==1.5 html5lib==0.9999999 parso==0.3.1 tornado==4.2 and then 6.1 pyzmq==17.1

To text file

Run

py -m pip download -d ./jup_env/jupyterinstall jupyter==1.0.0 -c constraints.txt

Copy directory and all files to VM

Install, install, install

I then cd to the Jupyter install directory and ran

py -m pip install --no-index --find-links=c:\jupyterinstall jupyter

but this repeatedly failed at various points either due to tornado 4.2, tornado 6.1 or argon-cffi

at this point I gave up.

I also tried extracting the notebook zip and running

Cd to extracted path

python setup.py install --single-version-externally-managed --root=/

But that failed also.

MsSQLGirl commented 3 years ago

@chlafreniere , @yualan , @markingmyname, @lucyzhang929 - FYI.

amtwo commented 3 years ago

We have been investing in SQL & PowerShell notebooks to create a robust troubleshooting Book that can be used by our large (~50 person) DBA team to use to support our production environment. Some servers are ONLY accessible from secure, internet-disconnected RDP "jump servers" and any troubleshooting tools must be installed in a completely offline manner on those servers.

Additionally, some of the team is outsourced contractors, who depend on accessing internet-disconnected jump servers as their primary domain-joined access. (Using internet-connected virtual desktops isn't an option due to internal/political reasons).

Getting ADS up and running by itself is an OK experience -- downloading the installer + a number of vsix packages is doable. However, getting PowerShell Notebooks to work is a seeming herculean feat. Similar to Mr Sewell's experience, the bird's nest of dependencies to get this minimally working has been (at best) discouraging, and frankly threatens the success of the solution for our team.

I have essentially done one-by-one "download, copy, install, try to open a notebook, parse errors, identify missing pip package, repeat" .... all the while attempting to keep track of the list of packages & versions so that we can automate this install on other jump servers. My concern is that even if I get this working, the maintenance & updates of all these python packages (50? 75? 100?) will be an untenable to maintain long-term.

sqllensman commented 3 years ago

After the upgrade to Version 1.30.0 the undisclosed requirement to upgrade to Python 3.8 resulted in my offline (working) version being ruined and being left in a state were Notebooks were again not usable.

As AzureData Studio is still not addressing the problems with installing in Offline or secure environments I will document below the steps I was able to take to actually get a working version. This is based on advice from https://stackoverflow.com/questions/11091623/how-to-install-packages-offline

First, on a machine that is open to Internet

  1. Download and Install Python 3.8.10 for Windows The is can be downloaded from https://www.python.org/downloads/windows/ This downloaded python-3.8.10-amd64.exe

In my case I installed to C:\Python\Python 3.8

  1. Open an elevated Cmd session and navigate to the scripts directory of the Python Installation eg C:\Python\Python 3.8\Scripts

  2. Check that pip is installed by running pip --version . This should return version of pip installed eg pip 21.1.1 from C:\Python\Python 3.8\lib\site-packages\pip <python 3.8>

  3. Run the command pip freeze > requirements.txt

This will create a list of required packages

  1. Create a sub directory Wheelhouse . can be done by running: mkdir wheelhouse
  2. Run command: pip download -r requirements.txt -d wheelhouse

This will download libs and their dependencies to directory wheelhouse

  1. Copy requirements.txt into wheelhouse directory
  2. Archive contents of wheelhouse into a compressed file The StackOverflow article suggested creating wheelhouse.tar.gz with tar -zcf wheelhouse.tar.gz wheelhouse but 7zip or other can be used

On the machine without Internet Access

  1. Copy azuredatastudio-windows-setup-1.30.0.exe python-3.8.10-amd64.exe wheelhouse.tar.gz (or wheelhouse.zip depending on what was used)

to a local directory

  1. Install azuredatastudio 1.30.0
  2. Install python-3.8.10 (I again installed to C:\Python\Python 3.8)
  3. Copy the wheelhouse.tar.gz to C:\Python\Python 3.8\Scripts and unzip to create a Wheelhouse directory
  4. From an elevated command prompt in C:\Python\Python 3.8\Scripts run pip install -r wheelhouse/requirements.txt --no-index --find-links wheelhouse

This will install the required packages onto the local Python Installation

  1. Start Azure Data Studio
  2. Create a new notebook and try to switch the Kernel to Python 3. This should launch the wizard to Configure Python to run Python 3 Kernel

Select Installation type of Use existing Python Installation and browse to C:\Python\Python 3.8 (or wherever you installed) image

Continue the wizard. This should complete as all the required files are installed

Both the Python 3 and PowerShell Kernel's should now work.