search

Esri / arcgis-python-api

Documentation and samples for ArcGIS API for Python
https://developers.arcgis.com/python/
Apache License 2.0
1.86k stars 1.09k forks source link

Requests to ArcGIS server send 2x when Portal secured with IWA #1869

Open PeaceNlove opened 1 month ago

PeaceNlove commented 1 month ago

Describe the bug Requests to ArcGIS Server admin are send 2x when the server is federated with Portal for ArcGIS, secured with IWA and the server webadaptor is not

To Reproduce Steps to reproduce the behavior:

1. I use gis._con.post to work around a edge case (so the bug could be coming from mee as well)
2. Set the loglevel to DEBUG
3. I noticed the request being send twice:
2024-07-12T12:44:45.6308277Z DEBUG:urllib3.connectionpool:https://myserver:443 "POST /gd1//admin/services/test HTTP/1.1" 200 230
2024-07-12T12:44:45.6401780Z DEBUG:urllib3.connectionpool:https:/myserver:443 "POST /gd1//admin/services/test HTTP/1.1" 200 230

error:

There is no error, but you can imagine the Server will behave weird as a response when you try to alter settings on the server

Screenshots

Expected behavior The request should be send only once

Platform (please complete the following information):

Additional context

This situation happened on a system of a client of mine. I don't have access to the system and cannot test there. They have a setup with Portal for ArcGIS secured with Web Tier Authentication on the Portal Webadapter and the ArcGIS Server Webadapter is not secured. As far as I could see, the ArcGIS Server did not have the hosting role.

I did some modifications to the connectionpool.py from urllib to let it dump full stack traces on each request. This showed for the first request: 2024-07-12T12:44:45.6326842Z res = self.gis._con.post(url,data, add_token=False, verify_cert=False,try_json=False, timeout=timeout) 2024-07-12T12:44:45.6327778Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1468, in post 2024-07-12T12:44:45.6328465Z resp = self._session.post( 2024-07-12T12:44:45.6329112Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\auth\api.py", line 525, in post 2024-07-12T12:44:45.6329760Z return self._session.post( 2024-07-12T12:44:45.6330406Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 637, in post 2024-07-12T12:44:45.6331170Z return self.request("POST", url, data=data, json=json, **kwargs) 2024-07-12T12:44:45.6331942Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 589, in request 2024-07-12T12:44:45.6332618Z resp = self.send(prep, **send_kwargs) 2024-07-12T12:44:45.6333305Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 703, in send 2024-07-12T12:44:45.6333980Z r = adapter.send(request, **kwargs) 2024-07-12T12:44:45.6334649Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\adapters.py", line 486, in send 2024-07-12T12:44:45.6335767Z resp = conn.urlopen( 2024-07-12T12:44:45.6336441Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 715, in urlopen 2024-07-12T12:44:45.6337235Z httplib_response = self._make_request( 2024-07-12T12:44:45.6337978Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 484, in _make_request 2024-07-12T12:44:45.6338654Z log.debug(full_stack())

and for the second request 2024-07-12T12:44:45.6420892Z res = self.gis._con.post(url,data, add_token=False, verify_cert=False,try_json=False, timeout=timeout) 2024-07-12T12:44:45.6421808Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1468, in post 2024-07-12T12:44:45.6422507Z resp = self._session.post( 2024-07-12T12:44:45.6423143Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\auth\api.py", line 525, in post 2024-07-12T12:44:45.6423787Z return self._session.post( 2024-07-12T12:44:45.6424445Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 637, in post 2024-07-12T12:44:45.6425191Z return self.request("POST", url, data=data, json=json, **kwargs) 2024-07-12T12:44:45.6426182Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 589, in request 2024-07-12T12:44:45.6426882Z resp = self.send(prep, **send_kwargs) 2024-07-12T12:44:45.6427559Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 710, in send 2024-07-12T12:44:45.6428269Z r = dispatch_hook("response", hooks, r, **kwargs) 2024-07-12T12:44:45.6429008Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\hooks.py", line 30, in dispatch_hook 2024-07-12T12:44:45.6429696Z _hook_data = hook(hook_data, **kwargs) 2024-07-12T12:44:45.6430485Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\auth\_auth\_winauth.py", line 182, in generate_portal_server_token 2024-07-12T12:44:45.6431406Z _r = r.connection.send(r.request, **kwargs) 2024-07-12T12:44:45.6432101Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\adapters.py", line 486, in send 2024-07-12T12:44:45.6432825Z resp = conn.urlopen( 2024-07-12T12:44:45.6433491Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 715, in urlopen 2024-07-12T12:44:45.6434193Z httplib_response = self._make_request( 2024-07-12T12:44:45.6434938Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 484, in _make_request 2024-07-12T12:44:45.6435983Z log.debug(full_stack())

The interesting part here is here 2024-07-12T12:44:45.6427559Z File "D:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 710, in send so apparently there is a hook and this hook is being called after the request has been send, which happens at line 703 However here is no reason to make this second request. The first one already succeeded. And i could argue the hook is being used the wrong way, because it is meant to modify the response after the request has been made and not to make another request of the same kind.

Since the hook is calling the generate_portal_server_token function, I found that this hook is coming from arcgis Auth.

In order to prevent the hook I saw the option for drop_auth on gis._con.post as optional boolean. However, it doesn't look like it's dropping the auth on post?

When I look at delete in _connection I see this: if drop_auth and self._session.auth: auth = self._session.auth self._session.auth = None resp = self._session.delete( url=url, data=params, allow_redirects=allow_redirects ) self._session.auth = auth else: resp = self._session.delete( url=url, data=params, allow_redirects=allow_redirects ) self._session.auth = None will drop the authentication and it's restored after the delete request has been made. The above code makes sense to me

However at post I see this: if self._session.auth and drop_auth: auth = self._session.auth self._session.auth else: auth = None

But it doesn't look like this code is actually dropping the auth? I would expect self._session.auth = None to be there as well. My solution: auth = self.gis._con._session.auth self.gis._con._session.auth = None res = self.gis._con.post(url,data, add_token=False, verify_cert=False,try_json=False, timeout=timeout) self.gis._con._session.auth = auth

Again: I do realize I'm using the gis._con object to do some stuff and normally I shouldn't do this. I would appriciate it if you could take a look at things I've found.

nanaeaubry commented 1 month ago

@jtroe @achapkowski Can you please help with this?

PeaceNlove commented 5 days ago

@nanaeaubry @achapkowski @jtroe any updates? Did you get a change to look at this?