Closed gswifort closed 1 month ago
Hi,
Currently, nothing is implemented to handle virtual environments. Each instance of AutoCAD fires up a new interpreter by calling Py_Initialize(), The module loads python312.dll into the AutoCAD process, python.exe is never called
I’m not entirely sure how to handle virtual environments in an embedded context, but I will look into this
Thanks for the answer. Yet on another topic. I haven't found an example of how to open a new empty document, add elements to it and save it. On various attempts I received:
RuntimeError:
Exception! (eWrongDatabase) in function appendAcDbEntity ,Line 2322, File PyDbSymbolTableRecord.cpp
Hi do you mean a new .DWG, as in a side database? Have a look at this sample
import traceback import PyRx as Rx import PyGe as Ge import PyGi as Gi import PyDb as Db import PyAp as Ap import PyEd as Ed #scope so everything is closed def processDb(db : Db.Database): line = Db.Line(Ge.Point3d(0,0,0),Ge.Point3d(100,100,0)) line.setDatabaseDefaults(db) model = Db.BlockTableRecord(db.modelSpaceId(), Db.OpenMode.kForWrite) model.appendAcDbEntity(line) def PyRxCmd_doit() -> None: try: sideDb = Db.Database() processDb(sideDb) sideDb.saveAs("e:\\newdwg.dwg") except Exception as err: traceback.print_exception(err)
Yes, that was it, thank you!
Coming back to the virtual environment, python312.dll
can always be the same, so it all comes down to loading <venv>\Lib\site-packages
into sys.path
instead of %localappdata%\programs\python\python312\Lib\site-packages
or at least site-packages
from a virtual environment before global ones
I think I found the solution, just set PYTHONPATH
to <venv>\Lib\site-packages
Great!
Note: I’m not happy with the current installer, I don’t want to have to write anything to the registry. So, I plan on writing a loader module that sets AutoCAD’s local ENV to the DLL paths, then loads the python wrapper module.
This loader module will expose PyConfig, https://docs.python.org/3/c-api/init_config.html#init-path-config , I’m not sure the format yet, maybe an .INI or XML configuration file that sits in the same folder.
This is some time off though, as I’m still writing wrappers
Yes, this should also solve the virtual environment issue (take into account the VIRTUAL_ENV
environment variable). Also keep in mind that AutoCAD can be launched using the COM interface (win32com.client.Dispatch
), and so far I have encountered a problem where AutoCAD does not see the modules located in the PyRxStubs
folder (I also solved this by adding to PYTHONPATH
, but this is a bit of a monkey-patching solution)
@gswifort
I have set up venv with launch.json
as below:
{
"version": "0.2.0",
"configurations": [
{
"name": "Remote Attach",
"type": "python",
"request": "attach",
"port": 5678,
"host": "127.0.0.1",
"justMyCode": false,
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "."
}
],
"env": {
"PYTHONPATH": "$PYTHONPATH:C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs"
},
}
]
}
VS Code states that env property is not allowed. I have tried adding PYTHONPATH="C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs"
to the .env
file. I still receive below error
Any hints?
Best regards
Sebastian
I have the following debugging settings:
launch.json
{
"name": "Autocad attach",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "127.0.0.1",
"port": 5678
},
"justMyCode": false
}
settings.json
{
"python.envFile": "${workspaceFolder}/.env"
}
.env
PYTHONPATH = c:\ProgramData\Autodesk\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs
However, the error you show will not be resolved by setting the PYTHONPATH to the "stubs" folder. The "stubs" folder contains only *.pyi
interfaces and Pyxx
modules (containing the source code) are only available in the AutoCAD environment.
Not quite sure how to deal with that vscode warning. The problem is PyLance can’t see python embedded inside C++ Those modules PyRx, PyGe etc.. are inside the .ARX file I’ve semi-resolved the issue by hiding it, see issue #3
If the problem is only the display of errors, you can simply ignore them, as follows:
import PyRx as Rx # type:ignore
import PyGe as Ge # type:ignore
import PyGi as Gi # type:ignore
import PyDb as Db # type:ignore
import PyAp as Ap # type:ignore
import PyEd as Ed # type:ignore
If you use isort then also:
import PyRx as Rx # isort:skip # type:ignore
import PyGe as Ge # isort:skip # type:ignore
import PyGi as Gi # isort:skip # type:ignore
import PyDb as Db # isort:skip # type:ignore
import PyAp as Ap # isort:skip # type:ignore
import PyEd as Ed # isort:skip # type:ignore
Thanks for the work. My current environment foresees the following working steps
1#Start BCAD and VS Code in parallel 2#Load script in BCAD manually via _PYLOAD 3#Start pydebug and call the remote debugger in VS Code 4#Alter and save script and restart from 2# via _PYLOAD (hopefully not killing BCAD)
Thus all modules must be installed accesible to the interpreter in BCAD (which is system-wide Python). VDEV is only available to VS Code. Is there any chance that I may have missed something and may directly launch and debug from within VS Code or reorganize my Python installation?
Best
Sebastian
For BricsCad to see the virtual environment (installed packages), the directory <virtualenv>\Lib\site-packages
must be added to sys.path
To do this you have two options:
# <virtualenv>\Scripts\activate.bat
...
set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;%PYTHONPATH%
and run BricsCad from the console with the virtual environment activated (for Autocad it is acad
, for BricsCad probably bcad
).
For me, when I run Autocad from a virtual environment, the embedded Python does not see the PyRxStubs
directory, so it also needs to be added to PYTHONPATH (You probably won't need this if you have the global PYTHONPATH
variable set correctly - it is set during installation):
# <virtualenv>\Scripts\activate.bat
...
set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;C:\ProgramData\Bricscad\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs;%PYTHONPATH%
# check if the path is correct, I only replaced Autocad → Bricscad.
>>> PYCMDPROMPT
>>> import sys
>>> sys.path.insert(0, ".\<virtualenv>\Lib\site-packages")
Some ideas:
-create loader modules for each host application. -The loader module will replace registry entries by adding env paths at runtime. -The loader module will be able to read a human editable configuration file so users can customize it.
-the installer will install main bulk of the package, i.e. wrappers, stubs, samples Users*username*\AppData\Local\Programs\PyRx
-Only the AutoCAD loader modules and PackageContents.xml will be installed in ApplicationPlugins -The loader modules for the clones will be installed in the PyRx folder where they can be added to a startup suite
What do you mean by loader module?
Another .ARX module that loads into Autocad, I'll need to hook into Autocads DLL paths and add the necessary paths
Hi, Can you test if v1.3.002 is enough to solve this task?
I’ve added these functions: PyPreConfig_InitPythonConfig PyConfig_InitPythonConfig
But I don’t quite understand how they are used, but the goal is to be able to add configurations to the INI, that I can pass along. But I don’t know which ones you might need
Hi, Wednesday at the earliest.
Hi, I don't know how to use what you did.
From what I see, I can use PyRx.ini
, but only globally, this file is not searched in the current directory (virtual environment).
For the virtual environment to work properly, it is necessary to adopt a different installation path scheme, as python.exe
does in the virtual environment (python3x.dll does not set paths in the same way as python.exe, it has to be done manually).
issue22213 and PEP 587 should be helpful.
Yeah, It clear I don’t understand how all this works.
“From what I see, I can use PyRx.ini, but only globally, this file is not searched in the current directory (virtual environment).”
It’s not meant to be, the design is to tell AutoCAD the path to the DLLs. And to pass information to Py_PreInitialize and Py_InitializeFromConfig, I just don’t have a clue what parameters are needed
How about we start with: PYTHONISOLATED = 1 PYTHONEXECUTABLE = C:\path\to\venv\Scripts\python.exe
I’ll pass these to Py_InitializeFromConfig
Yes, it is possible that this will be enough.
However, you should probably pay attention to one more thing - which python3x.dll
file to use. When creating a virtual environment, a pyvenv.cfg
file is created in the %VIRTUAL_ENV%
directory, containing, among others:
home = C:\Users\gswi\AppData\Local\Programs\Python\Python312
use the file located in this directory (we assume that the user may have different versions of python installed).
@gswifort is this design platform-independant?
Analyzing the python venv module also shows that it is platform-independent
And to determine whether to use an "isolated" configuration, I would check if the VIRTUAL_ENV
environment variable exists.
I’m preparing a build now; my thought is you must explicitly set PYTHONISOLATED in the INI I don’t want this to automatic, at least not yet
I installed the latest version (1.3.006), but I don't know how to configure the .ini
file.
I tested in various ways, but it does not see the virtual environment.
This is what the python.exe
configuration looks like in the virtual environment:
(venv) K:\Programy\demos\pyrx_demo>python -m sysconfig
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "venv"
Paths:
data = "K:\Programy\demos\pyrx_demo\venv"
include = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
platinclude = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
platlib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
platstdlib = "K:\Programy\demos\pyrx_demo\venv\Lib"
purelib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
scripts = "K:\Programy\demos\pyrx_demo\venv\Scripts"
stdlib = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"
Variables:
BINDIR = "\\pc-135\Users\GSWI\Documents\Programy\demos\pyrx_demo\venv\Scripts"
BINLIBDEST = "K:\Programy\demos\pyrx_demo\venv\Lib"
EXE = ".exe"
EXT_SUFFIX = ".cp312-win_amd64.pyd"
INCLUDEPY = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
LIBDEST = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"
TZPATH = ""
VERSION = "312"
VPATH = "..\.."
abiflags = ""
base = "K:\Programy\demos\pyrx_demo\venv"
exec_prefix = "K:\Programy\demos\pyrx_demo\venv"
installed_base = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
installed_platbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
platbase = "K:\Programy\demos\pyrx_demo\venv"
platlibdir = "DLLs"
prefix = "K:\Programy\demos\pyrx_demo\venv"
projectbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
py_version = "3.12.2"
py_version_nodot = "312"
py_version_nodot_plat = "312"
py_version_short = "3.12"
srcdir = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
userbase = "C:\Users\gswi\AppData\Roaming\Python"
and like this in autocad:
>>>: import sysconfig
>>>: sysconfig._main()
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "nt"
Paths:
data = "c:\users\gswi\appdata\local\programs\python\python312"
include = "c:\users\gswi\appdata\local\programs\python\python312\Include"
platinclude = "c:\users\gswi\appdata\local\programs\python\python312\Include"
platlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
platstdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
purelib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
scripts = "c:\users\gswi\appdata\local\programs\python\python312\Scripts"
stdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
Variables:
BINDIR = "C:\Program Files\Autodesk\AutoCAD 2022"
BINLIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
EXE = ".exe"
EXT_SUFFIX = ".cp312-win_amd64.pyd"
INCLUDEPY = "c:\users\gswi\appdata\local\programs\python\python312\Include"
LIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
TZPATH = ""
VERSION = "312"
VPATH = "..\.."
abiflags = ""
base = "c:\users\gswi\appdata\local\programs\python\python312"
exec_prefix = "c:\users\gswi\appdata\local\programs\python\python312"
installed_base = "c:\users\gswi\appdata\local\programs\python\python312"
installed_platbase = "c:\users\gswi\appdata\local\programs\python\python312"
platbase = "c:\users\gswi\appdata\local\programs\python\python312"
platlibdir = "DLLs"
prefix = "c:\users\gswi\appdata\local\programs\python\python312"
projectbase = "C:\Program Files\Autodesk\AutoCAD 2022"
py_version = "3.12.2"
py_version_nodot = "312"
py_version_nodot_plat = "312"
py_version_short = "3.12"
srcdir = "C:\Program Files\Autodesk\AutoCAD 2022"
userbase = "C:\Users\gswi\AppData\Roaming\Python"
Just of thought Instead of using the pycomand, try using the pyrx_onload.py in the bin directory
it loads early, so you might have to use a command or
import sysconfig
def OnPyLoadDwg(): print(sysconfig._main())
I'll try to set one up over the next week and try
Calling from the command level changes nothing.
The expected behavior is:
> python -m venv venv
> venv\Scripts\activate.bat
> acad
import PyRx as Rx # isort:skip
import PyGe as Ge # isort:skip
import PyGi as Gi # isort:skip
import PyDb as Db # isort:skip
import PyAp as Ap # isort:skip
import PyEd as Ed # isort:skip
def PyRxCmd_doit(): import sys print(*sys.path, sep="\n")
you should get:
...\AppData\Local\Programs\Python\Python312\python312.zip ...\AppData\Local\Programs\Python\Python312\DLLs ...\AppData\Local\Programs\Python\Python312\Lib ...\AppData\Local\Programs\Python\Python312 ...\venv # !!! ...\venv\Lib\site-packages # !!!
Note that one `.ini` file is not a good solution because we want to have different directories in `sys.path` depending on the environment we run autocad from.
I setup a venv
[PYRXSETTINGS] PYTHONINSTALLEDPATH =c:\users\dan\appdata\local\programs\python\python312 WXPYTHONPATH = M:\myenv\Lib\site-packages\wx PYRXSTUBPATH =C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs PYTHONISOLATED=1 PYTHONEXECUTABLE = M:\myenv\
output is
Regenerating model. Platform: "win-amd64" Python version: "3.12" Current installation scheme: "nt" Paths: data = "c:\users\dan\appdata\local\programs\python\python312" include = "c:\users\dan\appdata\local\programs\python\python312\Include" platinclude = "c:\users\dan\appdata\local\programs\python\python312\Include" platlib = "c:\users\dan\appdata\local\programs\python\python312\Lib\site-packages" platstdlib = "c:\users\dan\appdata\local\programs\python\python312\Lib" purelib = "c:\users\dan\appdata\local\programs\python\python312\Lib\site-packages" scripts = "c:\users\dan\appdata\local\programs\python\python312\Scripts" stdlib = "c:\users\dan\appdata\local\programs\python\python312\Lib" Variables: BINDIR = "M:\" BINLIBDEST = "c:\users\dan\appdata\local\programs\python\python312\Lib" EXE = ".exe" EXT_SUFFIX = ".cp312-win_amd64.pyd" INCLUDEPY = "c:\users\dan\appdata\local\programs\python\python312\Include" LIBDEST = "c:\users\dan\appdata\local\programs\python\python312\Lib" TZPATH = "" VERSION = "312" VPATH = "...." abiflags = "" base = "c:\users\dan\appdata\local\programs\python\python312" exec_prefix = "c:\users\dan\appdata\local\programs\python\python312" installed_base = "c:\users\dan\appdata\local\programs\python\python312" installed_platbase = "c:\users\dan\appdata\local\programs\python\python312" platbase = "c:\users\dan\appdata\local\programs\python\python312" platlibdir = "DLLs" prefix = "c:\users\dan\appdata\local\programs\python\python312" projectbase = "M:\" py_version = "3.12.2" py_version_nodot = "312" py_version_nodot_plat = "312" py_version_short = "3.12" srcdir = "M:\"
Yes, I had the same effect. I showed sysconfig
to illustrate where the differences are, what is more important is what paths we have in sys.path
.
Ok, I’ll goof around with it, over the next week.
Note that one .ini file is not a good solution
What’s your suggestion? We must know Python DLL path and wxPython dll path before loading otherwise AutoCAD will spit it out This is not going to be something that can change at runtime
If you want to create a separate ini
file, you can look for it in the directory where you run AutoCAD, but I recommend using the pyvenv.cfg
with home = C:\Users\ gswi\AppData\Local\Programs\Python\Python312
I had a validation error in my code, this seems correct now
[PYRXSETTINGS] PYTHONINSTALLEDPATH =c:\users\dan\appdata\local\programs\python\python312 WXPYTHONPATH =C:\path\to\myenv\Lib\site-packages\wx PYRXSTUBPATH =C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs PYTHONISOLATED=1 PYTHONEXECUTABLE = C:\path\to\myenv\Scripts\python.exe
PyRx version <1.3.006.20240805> loaded: Python Interpreter Loaded successfully! sys.executable=C:\path\to\myenv\Scripts\python.exe sys.prefix=C:\path\to\myenv sys.exec_prefix=C:\path\to\myenv sys.base_prefix=c:\users\dan\appdata\local\programs\python\python312 sys.base_exec_prefix=c:\users\dan\appdata\local\programs\python\python312 sys.path=c:\users\dan\appdata\local\programs\python\python312\python312.zip sys.path=c:\users\dan\appdata\local\programs\python\python312\DLLs sys.path=c:\users\dan\appdata\local\programs\python\python312\Lib sys.path=C:\Users\Dan\AppData\Local\Programs\Python\Python312 sys.path=C:\path\to\myenv sys.path=C:\path\to\myenv\Lib\site-packages sys.path=C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs sys.path=c:\users\dan\appdata\local\programs\pyrx\bin\
I had considered how to resolve the path to the .INI.
The search order for RxLoader25.0.arx to resolve PyRx25.0.arx is install dir then local. Resolution of PyRx.INI and pyrx_onload.py are always local to PyRx25.0.arx
I want to be able to switch environments easily for me to work in debug mode If you need different runtime configurations, you can copy those four files and put them where you like, and modify as you wish
I don’t quite understand this
home = C:\Users\ gswi\AppData\Local\Programs\Python\Python312
In my mind, I can’t resolve ‘home’ without loading python, I can’t load python without knowing the exact path to wxPython
Why can't you resolve "home" without loading python?
Hi, It’s not clear to me where HOME originates from, do you mean in the windows environment? If it’s in the windows environment, I can resolve it.
home
is located in the pyvenv.cfg
file generated by python when creating the virtual environment.
python -m venv venv_dir
When the virtual environment is activated, the VIRTUAL_ENV
environment variable is created.
project/
└── venv_dir/ (%VIRTUAL_ENV%)
├── pyvenv.cfg
└── ...
so you should load python from %home%\python312.dll
Still confused, how do I find pyvenv.cfg? Pretend your starting form a clean system, all you have is the path to python in PATH?
You can read the VIRTUAL_ENV
environment variable.
this is in the windows environment?
Yes
I went to my virtual environment folder and activated it I type in $Env:VIRTUAL_ENV and it’s empty
is this the right procedure?
The question is what does it mean that you activated them... :)
It is best to activate from the cmd console venv_dir\Scripts\activate.bat
I see, the batch sets it in that instance of PowerShell.
However it appears to be local, I’ll have to actually test if AutoCAD sees it
If you run AutoCAD from the console (type acad
in the console), it will see it.
Acad didn’t launch for me, but it sounds reasonable that would be the case. I will try to resolve VIRTUAL_ENV first, then fall back to the .INI for now
it sees it
Customization file loaded successfully. Customization Group: FEATUREDAPPS Regenerating model. true-C:\path\to\myenv AutoCAD menu utilities loaded.Cancel
Hi, I have a question: at what point does PyRx decide on the choice of a specific python interpreter? During installation or when loading into AutoCAD? I would like to run Autocad from a python virtual environment with a separate interpreter (e.g.
.\venv\Scripts\python.exe
), but PyRx in Autocad is always set to the global interpreter.