nautobot / nautobot

Network Source of Truth & Network Automation Platform
https://docs.nautobot.com
Apache License 2.0
1.05k stars 279 forks source link

'HTTP_PROXIES' not working for Git repository sync #1796

Closed TheRealBecks closed 2 years ago

TheRealBecks commented 2 years ago

Environment

Steps to Reproduce

Follow the standard installation process for an Ubuntu 20.04 server:

  1. https://nautobot.readthedocs.io/en/stable/installation/ubuntu/
  2. https://nautobot.readthedocs.io/en/stable/installation/nautobot/
  3. https://nautobot.readthedocs.io/en/stable/installation/services/
  4. https://nautobot.readthedocs.io/en/stable/installation/http-server/
  5. Open the browser and check that Nautobot is working --> Yes, works!
  6. Install the 'nautobot-welcome-wizard' --> Has been loaded
  7. Click 'Plugins' --> 'Welcome Wizard' --> Action 'Import' for 'Device Types' --> Site loaded correctly, but after refreshing the 'Import Device Types' table is still empty
  8. Click 'Extensibility' --> 'Git Repositories' --> 'Devicetype-library' --> 'Synchronization Status'
  9. After some time I will get the following two (timeout) errors. I did manual line breaks for easier reading:
Cmd('git') failed due to: exit code(128)
cmdline: git clone -v https://github.com/netbox-community/devicetype-library.git /opt/nautobot/git/devicetype_library
stderr: 'Cloning into '/opt/nautobot/git/devicetype_library'...
fatal: unable to access 'https://github.com/netbox-community/devicetype-library.git/':
Failed to connect to github.com port 443: Connection timed out '
Error while refreshing Devicetype-library:
Cmd('git') failed due to: exit code(128)
cmdline: git clone -v https://github.com/netbox-community/devicetype-library.git /opt/nautobot/git/devicetype_library
stderr: 'Cloning into '/opt/nautobot/git/devicetype_library'...
fatal: unable to access 'https://github.com/netbox-community/devicetype-library.git/':
Failed to connect to github.com port 443: Connection timed out '

Configuration

I configured my HTTP proxy in the file nautobot_config.py as follows, but replaced the real IP addresses:

# HTTP proxies Nautobot should use when sending outbound HTTP requests (e.g. for webhooks).
HTTP_PROXIES = {
     'http': 'http://192.0.2.1:3128',
     'https': 'http://192.0.2.1:3128',
}

Expected Behavior

I can see traffic flowing over the proxy server.

Observed Behavior

I used TCP dump on the proxy server and there we're no packets incoming from the Nautobot server. I also checked with tcpdump on the Nautobot server and can say, that it's sending it's HTTP(S) traffic to the default gateway, so no proxy has been used.

I also checked directly on the Linux console and I can clone repositories from Github. I also saw the traffic on the proxy server. --> Proy configuration in Linux is working

glennmatthews commented 2 years ago

Git operations use the GitPython library, which is a fairly thin wrapper around Git. You might try logging in to Linux as the nautobot user account and doing a git config --global https.proxy http://192.0.2.1:3128 command and see if that helps?

smk4664 commented 2 years ago

@glennmatthews beat me to it, but I came to the same conclusion.

TheRealBecks commented 2 years ago

Thanks a lot, it's working now! :) Run these commands as user nautobot:

git config --global http.proxy http://192.0.2.1:3128
git config --global https.proxy http://192.0.2.1:3128

It would be good if the git proxy configuration for Nautobot gets documented. That's the current HTTP_PROXIES documentation, but no word about the git commands. Do you think it's a good idea to do that?

glennmatthews commented 2 years ago

Absolutely that would be good to add to the documentation. Would you like to make a pull request?

TheRealBecks commented 2 years ago

@glennmatthews I created a pull request :smiley:

TheRealBecks commented 2 years ago

@glennmatthews Do I need to do anything to get that PR merged? Or will you guys do the job? :)

glennmatthews commented 2 years ago

We'll merge it shortly. Thank you! :-)