Closed mhw-at-yg closed 4 years ago
with 8af7ec70d15d62ae04180b4a495dba7b4b2dd724 I see that some dirs are missing from sys.path.
Here's the sys,path from archook after running the test script and the sys.path from a native Pro python session, with each sorted:
The native Pro path also includes it's own lib
and DLLs
sub-dirs which we don't have on the archook side.
I created branch fix-22
to work on this. Adding the rest of Pro's python folders to path changes the error from "circular import" to "no module named 'arcgisscripting'". My suspicion is that we can't cross major version numbers. My conda env is py3.8 and pro is py3.6.
$ python test\simple-test-arcpro.py
========================================
--- PRE sys.path:
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py3\python38.zip
C:\tools\miniconda3\envs\archook-py3\DLLs
C:\tools\miniconda3\envs\archook-py3\lib
C:\tools\miniconda3\envs\archook-py3
C:\tools\miniconda3\envs\archook-py3\lib\site-packages
d:\code-mhw-at-yg\archook
--- ArcGIS Pro archook result: None
--- POST sys.path:
C:\ArcGIS\Pro\Resources/ArcToolbox/Scripts
C:\ArcGIS\Pro\Resources/ArcPy
C:\ArcGIS\Pro\lib/site-packages
C:\ArcGIS\Pro\lib
C:\ArcGIS\Pro\DLLs
C:\ArcGIS\Pro\bin
C:\ArcGIS\Pro\
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py3\python38.zip
C:\tools\miniconda3\envs\archook-py3\DLLs
C:\tools\miniconda3\envs\archook-py3\lib
C:\tools\miniconda3\envs\archook-py3
C:\tools\miniconda3\envs\archook-py3\lib\site-packages
d:\code-mhw-at-yg\archook
*** Error: No module named 'arcgisscripting'
Something besides version numbers at play. With py27 and ArcMap I get "ImportError: DLL load failed: %1 is not a valid Win32 application.", which turns out also to hinge on arcgisscripting
.
$ python test\simple-test-arcmap.py
========================================
2.7.17 (default, Oct 28 2019, 21:35:07) [MSC v.1500 64 bit (AMD64)]
--- PRE sys.path:
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py2\python27.zip
C:\tools\miniconda3\envs\archook-py2\DLLs
C:\tools\miniconda3\envs\archook-py2\lib
C:\tools\miniconda3\envs\archook-py2\lib\plat-win
C:\tools\miniconda3\envs\archook-py2\lib\lib-tk
C:\tools\miniconda3\envs\archook-py2
C:\Users\mhwilkie\AppData\Roaming\Python\Python27\site-packages
C:\tools\miniconda3\envs\archook-py2\lib\site-packages
d:\code-mhw-at-yg\archook
--- ArcGIS Desktop archook result: None
--- POST sys.path:
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py2\python27.zip
C:\tools\miniconda3\envs\archook-py2\DLLs
C:\tools\miniconda3\envs\archook-py2\lib
C:\tools\miniconda3\envs\archook-py2\lib\plat-win
C:\tools\miniconda3\envs\archook-py2\lib\lib-tk
C:\tools\miniconda3\envs\archook-py2
C:\Users\mhwilkie\AppData\Roaming\Python\Python27\site-packages
C:\tools\miniconda3\envs\archook-py2\lib\site-packages
d:\code-mhw-at-yg\archook
C:\ArcGIS\Desktop10.7\arcpy
C:\ArcGIS\Desktop10.7\bin
C:\ArcGIS\Desktop10.7\ArcToolbox\Scripts
*** Error: DLL load failed: %1 is not a valid Win32 application.
Traceback (most recent call last):
File "test\simple-test-arcmap.py", line 21, in <module>
import arcpy
File "C:\ArcGIS\Desktop10.7\arcpy\arcpy\__init__.py", line 22, in <module>
from arcpy.geoprocessing import gp
File "C:\ArcGIS\Desktop10.7\arcpy\arcpy\geoprocessing\__init__.py", line 14, in <module>
from _base import *
File "C:\ArcGIS\Desktop10.7\arcpy\arcpy\geoprocessing\_base.py", line 14, in <module>
import arcgisscripting
ImportError: DLL load failed: %1 is not a valid Win32 application.
I believe you need to use a 32-bit interpreter with ArcGIS Desktop, it looks like you are using a 64-bit interpreter.
My suspicion is that we can't cross major version numbers. My conda env is py3.8 and pro is py3.6.
I believe that is correct as well, major/minor must match, e.g. use Python 3.6 if Pro's version of Python is 3.6.
thanks @logan-pugh you are right, with 32bit py2.7 it works. So now to add that check to the module...
Could just make a note in the readme. Technically there may also be a 64-bit version of arcpy installed with ArcGIS Desktop if you installed Background Geoprocessing (64-bit), but that isn't something I've tried to target.
Python 2.7 is believe resolved in fix-22 branch, as of ad6026b8d9.
test\create-conda-py27-32-env.bat
Python 3.6 (x64) and Pro continues to fail. Same error as above. Env created with test\create-conda-py36-env.bat
. So now we know it's not bitness or python version mismatch. I don't know where to look next.
Py3.6 and Pro test result:
$ python test\simple-test-arcpro.py
========================================
3.6.10 |Anaconda, Inc.| (default, Mar 23 2020, 17:58:33) [MSC v.1916 64 bit (AMD64)]
--- PRE sys.path:
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py36\python36.zip
C:\tools\miniconda3\envs\archook-py36\DLLs
C:\tools\miniconda3\envs\archook-py36\lib
C:\tools\miniconda3\envs\archook-py36
C:\tools\miniconda3\envs\archook-py36\lib\site-packages
d:\code-mhw-at-yg\archook
--- ArcGIS Pro archook result: None
--- POST sys.path:
C:\ArcGIS\Pro\Resources/ArcToolbox/Scripts
C:\ArcGIS\Pro\Resources/ArcPy
C:\ArcGIS\Pro\lib/site-packages
C:\ArcGIS\Pro\lib
C:\ArcGIS\Pro\DLLs
C:\ArcGIS\Pro\bin
C:\ArcGIS\Pro\
D:\code-mhw-at-yg\archook\test
C:\tools\miniconda3\envs\archook-py36\python36.zip
C:\tools\miniconda3\envs\archook-py36\DLLs
C:\tools\miniconda3\envs\archook-py36\lib
C:\tools\miniconda3\envs\archook-py36
C:\tools\miniconda3\envs\archook-py36\lib\site-packages
d:\code-mhw-at-yg\archook
*** Error: No module named 'arcgisscripting'
Traceback (most recent call last):
File "test\simple-test-arcpro.py", line 21, in <module>
import arcpy
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\__init__.py", line 74, in <module>
from arcpy.geoprocessing import gp
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\geoprocessing\__init__.py", line 14, in <module>
from ._base import *
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\geoprocessing\_base.py", line 14, in <module>
import arcgisscripting
ModuleNotFoundError: No module named 'arcgisscripting'
Try using a vanilla Python installation rather than conda, to see if it's something specific to using conda.
Test with portable WinPython v3.6.8, x64 zero edition, with numpy installed by pip after unzipping. I currently have ArcGIS Pro 2.5.2208 installed. I think I had 2.4 when issue first opened.
D:\code-mhw-at-yg\archook>python test\simple-test-arcpro.py
========================================
3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]
--- PRE sys.path:
D:\code-mhw-at-yg\archook\test
C:\tools\WPy64-3680\python-3.6.8.amd64\python36.zip
C:\tools\WPy64-3680\python-3.6.8.amd64\DLLs
C:\tools\WPy64-3680\python-3.6.8.amd64\lib
C:\tools\WPy64-3680\python-3.6.8.amd64
C:\tools\WPy64-3680\python-3.6.8.amd64\lib\site-packages
d:\code-mhw-at-yg\archook
--- ArcGIS Pro archook result: None
--- POST sys.path:
C:\ArcGIS\Pro\Resources/ArcToolbox/Scripts
C:\ArcGIS\Pro\Resources/ArcPy
C:\ArcGIS\Pro\lib/site-packages
C:\ArcGIS\Pro\lib
C:\ArcGIS\Pro\DLLs
C:\ArcGIS\Pro\bin
C:\ArcGIS\Pro\
D:\code-mhw-at-yg\archook\test
C:\tools\WPy64-3680\python-3.6.8.amd64\python36.zip
C:\tools\WPy64-3680\python-3.6.8.amd64\DLLs
C:\tools\WPy64-3680\python-3.6.8.amd64\lib
C:\tools\WPy64-3680\python-3.6.8.amd64
C:\tools\WPy64-3680\python-3.6.8.amd64\lib\site-packages
d:\code-mhw-at-yg\archook
arcpy must be run from inside the Conda environment, which ArcGIS
uses to manage the installation of Python.
You can access this environment by launching Python from the links inside the
"ArcGIS > ArcGIS Pro" Start Menu Group:
"Python Interactive Terminal" starts a Python session.
"Python Command Prompt" opens a Command Prompt window initialized with Conda.
Or by running the proenv.bat file in the terminal:
C:\> C:\ArcGIS\Pro\bin\python\scripts\proenv.bat
*** Error: arcpy needs to run within an active ArcGIS Conda environment
Traceback (most recent call last):
File "test\simple-test-arcpro.py", line 21, in <module>
import arcpy
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\__init__.py", line 70, in <module>
raise ImportError("arcpy needs to run within an active ArcGIS Conda environment")
ImportError: arcpy needs to run within an active ArcGIS Conda environment
ImportError: arcpy needs to run within an active ArcGIS Conda environment
This is noted in the README -- you'll need to create a conda-meta
directory in your interpreter's root directory, arcpy has a dumb check that that folder exists regardless of whether it's actually using conda.
thanks for that @logan-pugh, I had missed that obvious one. It's not the issue though, creating that folder reverts to same arcgisscripting error for me.
$ mkdir C:\tools\WPy64-3680\python-3.6.8.amd64\conda-meta
mhwilkie@ENV-Y225411 D:\code\public\archook
$ python test\simple-test-arcpro.py
========================================
3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]
--- PRE sys.path:
D:\code\public\archook\test
C:\tools\WPy64-3680\python-3.6.8.amd64\python36.zip
C:\tools\WPy64-3680\python-3.6.8.amd64\DLLs
C:\tools\WPy64-3680\python-3.6.8.amd64\lib
C:\tools\WPy64-3680\python-3.6.8.amd64
C:\tools\WPy64-3680\python-3.6.8.amd64\lib\site-packages
d:\code\public\archook
--- ArcGIS Pro archook result: None
--- POST sys.path:
C:\ArcGIS\Pro\Resources/ArcToolbox/Scripts
C:\ArcGIS\Pro\Resources/ArcPy
C:\ArcGIS\Pro\lib/site-packages
C:\ArcGIS\Pro\lib
C:\ArcGIS\Pro\DLLs
C:\ArcGIS\Pro\bin
C:\ArcGIS\Pro\
D:\code\public\archook\test
C:\tools\WPy64-3680\python-3.6.8.amd64\python36.zip
C:\tools\WPy64-3680\python-3.6.8.amd64\DLLs
C:\tools\WPy64-3680\python-3.6.8.amd64\lib
C:\tools\WPy64-3680\python-3.6.8.amd64
C:\tools\WPy64-3680\python-3.6.8.amd64\lib\site-packages
d:\code\public\archook
*** Error: No module named 'arcgisscripting'
Traceback (most recent call last):
File "test\simple-test-arcpro.py", line 21, in <module>
import arcpy
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\__init__.py", line 74, in <module>
from arcpy.geoprocessing import gp
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\geoprocessing\__init__.py", line 14, in <module>
from ._base import *
File "C:\ArcGIS\Pro\Resources/ArcPy\arcpy\geoprocessing\_base.py", line 14, in <module>
import arcgisscripting
ModuleNotFoundError: No module named 'arcgisscripting'
Haven't tried WinPython myself, I've only used the official release (3.6.8 x86-64) with archook.
That said I still suspect either an issue with sys.path
or os.environ['PATH']
, so I would take a closer look at those and compare the environment set up by proenv.bat
vs. what archook does.
For one thing, some of the directories in your sys.path
output above likely do not exist:
C:\ArcGIS\Pro\lib/site-packages C:\ArcGIS\Pro\lib C:\ArcGIS\Pro\DLLs
They should rather point to (respectively):
<ArcGIS Pro install dir>\bin\Python\envs\arcgispro-py3\lib\site-packages
<ArcGIS Pro install dir>\bin\Python\envs\arcgispro-py3\lib
<ArcGIS Pro install dir>\bin\Python\envs\arcgispro-py3\DLLs
That would definitely cause arcgisscripting to not be able to be located.
Thanks again Logan. I definitely had tunnel vision on this one. I'm embarrassed I didn't think to do the obvious of seeing if the dirs existed or comparing with Pro's native path settings!
The new code works with WinPython and Conda python py3.6 64bit, for me. I'm going to close this long winded issue and open new ones for anything that might arise out of the fixes (for instance I'm not sure py2.7 works at the moment).
Recipe: