Question about Using a Proxy Server with ArcREST Samples

[RandySincoular]

ArcRest or ArcRestHelper

Both?

Version or date of download

18-Apr-16

### Bug or Enhancement Question: I have a script based on the adds_rows.py sample that reads records from a local FGDB and is supposed to update a feature layer we have in AGOL. This script works from a network connection outside of our company, but inside our network the script fails.

Our firewall group is telling me that the script is not using our proxy server but going directly to our firewall where it is being blocked. They also said that there appears to be traffic taking two paths to get to the internet. One path is going through our firewall and another is trying to go through our proxy. Not sure if I understand this. they have checked firewall rules, run firewall debug sessions, etc. No luck so far.

Is there something else I should be doing to set the proxy_url or am I missing something else? Or, is there something else our firewall group should be looking at?

Thanks,

Randy

Repo Steps or Enhancement details

""" This sample shows how to add rows from a FGDB to an AGOL layer

""" import arcrest from arcresthelper import featureservicetools from arcresthelper import common

def trace(): """ trace finds the line, the filename and error message and returns it to the user """ import traceback, inspect,sys tb = sys.exc_info()[2] tbinfo = traceback.format_tb(tb)[0] filename = inspect.getfile(inspect.currentframe())

script name + line number

line = tbinfo.split(", ")[1]
# Get Python syntax error
#
synerror = traceback.format_exc().splitlines()[-1]
return line, filename, synerror

def main():

Use these settings when running inside of XXXXX Network

proxy_port = 80
proxy_url = 'msn-proxyxxxxxxx.com'

# Use these settings when running outside of XXXX Energy
# proxy_port = None
# proxy_url = None

# Use these settings with testing with Fiddler Test Proxy (Tyler set this up to troubleshoot a proxy issue)

# proxy_port = 8888
#proxy_url = 'localhost'

securityinfo = {}
securityinfo['security_type'] = 'Portal'#LDAP, NTLM, OAuth, Portal, PKI
securityinfo['username'] = "Randy"#<UserName>
securityinfo['password'] = "xxxxxxx"#<Password>
securityinfo['org_url'] = "http://www.arcgis.com"
securityinfo['proxy_url'] = proxy_url
securityinfo['proxy_port'] = proxy_port
securityinfo['referer_url'] = None
securityinfo['token_url'] = None
securityinfo['certificatefile'] = None
securityinfo['keyfile'] = None
securityinfo['client_id'] = None
securityinfo['secret_id'] = None

itemId = "9f59af52c6474b61bc561bd6096b8ef7"#<Item ID>

# Get error: URLError: <urlopen error [Errno 10060]
layerName = "http://services3.arcgis.com/sqVn1QnQEPnGQy9r/arcgis/rest/services/AEOutageData2/FeatureServer/0"

# Path to Local FGDB Featureclass on PC
pathToFeatureClass = r"C:\gis\AEOutageData2.gdb\OutageEvent"

try:

    fst = featureservicetools.featureservicetools(securityinfo)

    if fst.valid == False:
        print fst.message
    else:
        print "calling AddFeatures ..."

        results = fst.AddFeaturesToFeatureLayer(layerName,pathToFeatureClass,chunksize=2000)

        print "done adding feature"

except (common.ArcRestHelperError),e:
    print "error in function: %s" % e[0]['function']
    print "error on line: %s" % e[0]['line']
    print "error in file name: %s" % e[0]['filename']
    print "with error message: %s" % e[0]['synerror']
    if 'arcpyError' in e[0]:
        print "with arcpy message: %s" % e[0]['arcpyError']

except:
    line, filename, synerror = trace()
    print "error on line: %s" % line
    print "error in file name: %s" % filename
    print "with error message: %s" % synerror

if name == "main": main()

*There are 2 features in the local FGDB.

Error:

* Remote Interpreter Reinitialized *

calling AddFeatures ... 2 features in layer error in function: AddFeaturesToFeatureLayer error on line: line 510 error in file name: C:\Python27\ArcGIS10.2\lib\site-packages\arcresthelper\featureservicetools.py with error message: URLError: <urlopen error [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>

[BrunoCaimar]

@RandySincoular I've faced similar problems using ArcREST inside networks with proxy as well. I'm using a workaround here forcing proxy through Python environment variables.

Here is the code that I'm using:

def set_proxy_environment_variables():
    proxies = urllib.getproxies()
    http_proxy = proxies.get('http')
    https_proxy = proxies.get('https')

    Utils.logs("Proxies: {0}".format(proxies))

    if http_proxy is not None and http_proxy != "":
        Utils.logs("Configurando http proxy: {0}".format(http_proxy))
        os.environ["HTTP_PROXY"] = http_proxy

    if https_proxy is not None and https_proxy != "":
        Utils.logs("Configurando https proxy: {0}".format(https_proxy))
        os.environ["HTTPS_PROXY"] = https_proxy

I hope it helps.

[RandySincoular]

Thanks Bruno, that should do the trick. I appreciate your help.

Randy

From: Bruno Caimar [mailto:notifications@github.com] Sent: Friday, April 29, 2016 1:53 PM To: Esri/ArcREST Cc: Sincoular, Randy; Mention Subject: Re: [Esri/ArcREST] Question about Using a Proxy Server with ArcREST Samples (#228)

@RandySincoularhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_RandySincoular&d=CwMCaQ&c=GUDVeAVg1gjs_GJkmwL1m3gEzDND7NeJG5BIAX_2yRE&r=VPwuEa_rNmp_UVm0od-YvdtKnlNWqqwbpxa2Pz1fNVE&m=ssSqFGImqpL14ItF60bQ9aXxBgECVpzGAWSt-IHfBbo&s=wl3fSMSkNEBNpLiRfkEURXLThbVLRmOcfIra_Muhw8M&e= I've faced similar problems using ArcREST inside networks with proxy as well. I'm using a workaround here forcing proxy through Python environment variables.

Here is the code that I'm using:

def set_proxy_environment_variables():

proxies = urllib.getproxies()

http_proxy = proxies.get('http')

https_proxy = proxies.get('https')

Utils.logs("Proxies: {0}".format(proxies))

if http_proxy is not None and http_proxy != "":

    Utils.logs("Configurando http proxy: {0}".format(http_proxy))

    os.environ["HTTP_PROXY"] = http_proxy

if https_proxy is not None and https_proxy != "":

    Utils.logs("Configurando https proxy: {0}".format(https_proxy))

    os.environ["HTTPS_PROXY"] = https_proxy

I hope it helps.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_ArcREST_issues_228-23issuecomment-2D215847222&d=CwMCaQ&c=GUDVeAVg1gjs_GJkmwL1m3gEzDND7NeJG5BIAX_2yRE&r=VPwuEa_rNmp_UVm0od-YvdtKnlNWqqwbpxa2Pz1fNVE&m=ssSqFGImqpL14ItF60bQ9aXxBgECVpzGAWSt-IHfBbo&s=zhLjYnhDyAEqXRBetQo1rHnGFxqKX4mCV0or35LjVa4&e=

[ScrollLock]

Hello,

May I know in what part of the script should we put it? As you see in the script provided by Randy that it uses specific variable namely the proxy_port and proxy_url ?

We are also working with arcRest and we are encountering the 407 Proxy Authentication Required error. I already setup the HTTPS_PROXY and HTTP_PROXY in the environment variables and we using the user's windows id.

Any help would be very much appreciated.

[BrunoCaimar]

Hi @ScrollLock,

You have to set the proxy variables before any call to ArcREST. Usually I do that at the first thing on my scripts.

You can try to set proxy url with the username/password to try to bypass the 407 error message. It will be something like:

os.environ["HTTP_PROXY"] = "DOMAIN\User_Name:Passw0rd123@PROXY_SERVER_NAME_OR_IP:PORT"
os.environ["HTTPS_PROXY"] = "DOMAIN\User_Name:Passw0rd123@PROXY_SERVER_NAME_OR_IP:PORT"

Using http with Python on Windows it's a bit annoying. Sometimes it works without user/pwd and sometimes it does not.

Bruno