Question: Token acceptance

[WYMcCunE]

Problem: We are running this script on an FME server (FME Flow), and therefore not explicitly logged into the portal via ArcPro We confirmed that within the FME PythonCaller we can SignInToPortal / get a token, but this is not persisting to the NetworkAnalysis.

Below traceback is run locally (FME Form) when I am not logged in to the portal via arcpro, and reproducible on FME Flow

The LargeNetworkAnalysis tool only accepts the url, and in the parallel processing the login is not recognized with the SignInToPortal (I hypothesize this is because of an env to env issue)

Context: our credentials expire every 12 hours so logging in the portal via the server manually is not feasible. The ideal would be to pass a token or client id to the function as we have the credentials, but no way to pass them to the parallel processes. We tried passing the output from arcgis.gis.GIS to the network analysis the Network_Data_Source kwarg, but this was rejected Is there a way to pass a token to the network analysis?

Validating OD Cost Matrix settings...
Getting tool limits from the portal...
Inputs successfully validated.
Copying input origins and destinations to outputs...
Using ExportFeatures
Spatially sorting input dataset OutOrigins
Spatially sorting input dataset OutDestinations
Intermediate outputs will be written to C:\Users\WYMcCuneE\AppData\Local\Temp\2\wbrun_1718904936427_10396\fmetmp_6\scratch\ODCM_73939de13d924bdc9f61b71810d5a08e.
Error initializing OD Cost Matrix analysis.
ERROR: Traceback (most recent call last):
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 763, in _validate_od_settings
ERROR:     odcm.initialize_od_solver()
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 154, in initialize_od_solver
ERROR:     self.od_solver = arcpy.nax.OriginDestinationCostMatrix(self.network_data_source)
ERROR:   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\_na\_odcms.py", line 82, in __init__
ERROR:     super().__init__(in_network)
ERROR: ValueError: Not signed into Portal '<portal_url>'.
Error in parallelization subprocess.
ERROR: Traceback (most recent call last):
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 1108, in launch_parallel_od
ERROR:     od_calculator.solve_od_in_parallel()
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 790, in solve_od_in_parallel
ERROR:     self.optimized_cost_field = self._validate_od_settings()
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 763, in _validate_od_settings
ERROR:     odcm.initialize_od_solver()
ERROR:   File "\\servername\FME_Flow_Share\resources\engine\Plugins\Python\python39\large-network-analysis-tools\parallel_odcm.py", line 154, in initialize_od_solver
ERROR:     self.od_solver = arcpy.nax.OriginDestinationCostMatrix(self.network_data_source)
ERROR:   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\_na\_odcms.py", line 82, in __init__
ERROR:     super().__init__(in_network)
ERROR: ValueError: Not signed into Portal '<portal_url>'.
ERROR: Parallelization using parallel_odcm.py failed.

[mmorang]

This is a really interesting question. I have no idea what the solution is off the top of my head, but I'll consult with some colleagues and get back to you.

[mmorang]

Sorry for the lack of response here. We're still trying to make sense of what's going on.

Here are some thoughts and ideas based on what I think I understand so far:

The tool itself doesn't do anything to sign in to the portal being used. It inherits the state from the caller (in your case, FME). This seems to be a problem if running the tool takes a very long time because your login expires, and the tool doesn't do anything to refresh it.

A simple/hacky solution would be to just add a call to arcpy.SignInToPortal() somewhere within the part of the tool code that does the OD cost matrix analyses in parallel. I suggest adding an else statement here: https://github.com/Esri/large-network-analysis-tools/blob/master/parallel_odcm.py#L122-L124

        if not self.is_service:
            self._make_nds_layer()
        else:
            arcpy.SignInToPortal(username, password)

I had thought that SignInToPortal requires you to put in your username and password in plain text, but it looks like they have enhanced it to also work with tokens. So maybe depending on how you're doing this, you could check the current token's expiry time, and generate a new one if needed, all in that area of code.

Note that SignInToPortal only works with certain types of authentication for Enterprise portals. But if you're using ArcGIS Online, that probably won't matter.

Does this help you at all?

[WYMcCunE]

Hi @mmorang thanks for looking into this; We did modify the code starting at line 703 in helpers.py in a similar fashion; with your suggestion, we don't have a way to pass username and password to the tool from FME, or a token. So, we are storing the creds in the server itself for access, and this is working so far.

It is preferential to not reference username/password in code whatsoever, to reduce the chances of a password ever appearing in a log or traceback.

I could not get SignInToPortal to accept a token ; it would raise a "Token is not a valid kwarg" ; we are using Esri Python 3.9.

If we implemented the signintoportal where you suggest, where should the token exist / be passed? We noticed that in the FME Python caller environment could be signed in to the portal, but this access wouldn't persist to the parallel jobs environments.

I will DM you the snippet of our workaround