Adapt upgrade.sh for offline systems

[thefreakquency]

Issue type

[ x ] Feature request

Environment

• Python version: 3.5.4
• NetBox version: 2.2.7

Description

When running the upgrade.sh script on a system that does not have internet access, it can take several minutes to complete due do checks made by pip for every item in the requirements.txt file.

root@HOST:/opt/netbox# ./upgrade.sh Running NetBox upgrade as root, press any key to continue or ^C to cancel Cleaning up stale Python bytecode (find . -name "*.pyc" -delete)... Removing old Python packages (pip3 uninstall -r old_requirements.txt -y)... Cannot uninstall requirement pycrypto, not installed Updating required Python packages (pip3 install -r requirements.txt --upgrade)... Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fbe00f98a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/django/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fbe00f98080>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/django/ Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fbe00f984e0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/django/ Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionErr or('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fbe018cc400>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/django/ Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fbe018cc550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/django/ Requirement already up-to-date: Django<2.0,>=1.11 in /usr/local/lib/python3.5/dist-packages (from -r requirements.txt (line 1))

And this goes on an on and on... for over an hour!!!

Although mentioned to install "any new Python packages", the following script section tries to upgrade any package listed in the requirements.txt file even if the required version is met:

# Install any new Python packages
COMMAND="${PREFIX}${PIP} install -r requirements.txt --upgrade"
echo "Updating required Python packages ($COMMAND)..."
eval $COMMAND

Some kind of magic could happen and warn if an item in the requirement is not met without trying to upgrade all of the 20 items to the latest and greatest version...

I see the following possible approaches:

• Leveraging py.test from the python-pytest package (https://pypi.python.org/pypi/pytest-reqs), but this would require a pre-check for this module before using it, not ideal.
• Adding steps to the upgrade.sh script to validate the output of pip freeze -r requirements.txt against requirements.txt and then warn the user and halt if a dependency is not met.
• Adapt a python script (see https://github.com/pypa/pip/issues/2733) and ensure it works on both py2 & py3

This whole behavior could be activated by an "-offline" switch if not required by the majority of users, or an automated check to the outside world: wget -q --tries=10 --timeout=20 --spider http://google.com if [[ $? -eq 0 ]]; then echo "Online" else echo "Offline" fi

[jeremystretch]

The upgrade.sh script is provided as a convenience and works for most of the user base. There will always be a subset of users for whom it doesn't work due to constraints placed on their environments. Extending the upgrade script to satisfy unknown constraints (lack of Internet access, reduced filesystem permissions, archiving the old installation, etc.) is outside the scope of the NetBox project. Fortunately, it is trivial to adapt the stock script to conduct an upgrade as appropriate for your specific environment.