civicmapper / culvert-toolkit

GIS tools designed to support analysis of culvert performance within their watershed
https://civicmapper.github.io/culvert-toolkit/
Apache License 2.0
0 stars 0 forks source link

scratch workspace detection issues #35

Open gassc opened 11 months ago

gassc commented 11 months ago

Problem

We encountered an error with calculator execution right off the bat like this:

Traceback (most recent call last):
  File "<string>", line 111, in execute
  File "C:\Users\User.Name\AppData\Local\ESRI\conda\envs\CornellCulvertTool\Lib\site-packages\drainit\workflows.py", line 543, in __init__
    self.save_config_json_filepath = f'{self.gp._so("drainit_config", suffix="", where="folder")}.json'
  File "C:\Users\User.Name\AppData\Local\ESRI\conda\envs\CornellCulvertTool\Lib\site-packages\drainit\services\gp\_esri\__init__.py", line 171, in _so
    location = Path(env.scratchFolder)
  File "C:\Users\User.Name\AppData\Local\ESRI\conda\envs\CornellCulvertTool\Lib\pathlib.py", line 1082, in __new__
    self = cls._from_parts(args, init=False)
  File "C:\Users\User.Name\AppData\Local\ESRI\conda\envs\CornellCulvertTool\Lib\pathlib.py", line 707, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "C:\Users\User.Name\AppData\Local\ESRI\conda\envs\CornellCulvertTool\Lib\pathlib.py", line 691, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Failed to execute (CulvertCapacityPytTool).

The error was that arcpy.env.scratchFolder was returning None. This folder is relied on for temporary file i/o.

Cause

It looks like the global environment setting in ArcGIS Pro for the scratch workspace has to be set explicitly to a folder. That setting is here:

image

It can not be set using a Model Builder-style variable, e.g., %my_scratch_folder%.

That was the issue in this particular case.

Solution

For end-users

Set the scratch workspace explicitly. It's not clear if model builder variables are supported as global environment settings.

In the codebase

We should fall back to the system temp directory if the configured ArcGIS Pro scratch folder/workspace isn't available for some reason. That change would happen in the scratch output path generator function:

https://github.com/civicmapper/culvert-toolkit/blob/0542b1e238474578d149dc7a5cecef0e56205163/src/drainit/services/gp/_esri/__init__.py#L136