Python virtual environment

[gswifort]

Hi, I have a question: at what point does PyRx decide on the choice of a specific python interpreter? During installation or when loading into AutoCAD? I would like to run Autocad from a python virtual environment with a separate interpreter (e.g. .\venv\Scripts\python.exe), but PyRx in Autocad is always set to the global interpreter.

[CEXT-Dan]

Hi,

Currently, nothing is implemented to handle virtual environments. Each instance of AutoCAD fires up a new interpreter by calling Py_Initialize(), The module loads python312.dll into the AutoCAD process, python.exe is never called

I’m not entirely sure how to handle virtual environments in an embedded context, but I will look into this

[gswifort]

Thanks for the answer. Yet on another topic. I haven't found an example of how to open a new empty document, add elements to it and save it. On various attempts I received:

RuntimeError:
Exception! (eWrongDatabase) in function appendAcDbEntity ,Line 2322, File PyDbSymbolTableRecord.cpp

[CEXT-Dan]

Hi do you mean a new .DWG, as in a side database? Have a look at this sample

import traceback
import PyRx as Rx
import PyGe as Ge
import PyGi as Gi
import PyDb as Db
import PyAp as Ap
import PyEd as Ed

#scope so everything is closed
def processDb(db : Db.Database):
    line = Db.Line(Ge.Point3d(0,0,0),Ge.Point3d(100,100,0))
    line.setDatabaseDefaults(db)
    model = Db.BlockTableRecord(db.modelSpaceId(), Db.OpenMode.kForWrite)
    model.appendAcDbEntity(line)

def PyRxCmd_doit() -> None:
    try:
        sideDb = Db.Database()
        processDb(sideDb)
        sideDb.saveAs("e:\\newdwg.dwg")
    except Exception as err:
        traceback.print_exception(err)

[gswifort]

Yes, that was it, thank you!

Coming back to the virtual environment, python312.dll can always be the same, so it all comes down to loading <venv>\Lib\site-packages into sys.path instead of %localappdata%\programs\python\python312\Lib\site-packages or at least site-packages from a virtual environment before global ones

[gswifort]

I think I found the solution, just set PYTHONPATH to <venv>\Lib\site-packages

[CEXT-Dan]

Great!

Note: I’m not happy with the current installer, I don’t want to have to write anything to the registry. So, I plan on writing a loader module that sets AutoCAD’s local ENV to the DLL paths, then loads the python wrapper module.

This loader module will expose PyConfig, https://docs.python.org/3/c-api/init_config.html#init-path-config , I’m not sure the format yet, maybe an .INI or XML configuration file that sits in the same folder.

This is some time off though, as I’m still writing wrappers

[gswifort]

Yes, this should also solve the virtual environment issue (take into account the VIRTUAL_ENV environment variable). Also keep in mind that AutoCAD can be launched using the COM interface (win32com.client.Dispatch), and so far I have encountered a problem where AutoCAD does not see the modules located in the PyRxStubs folder (I also solved this by adding to PYTHONPATH, but this is a bit of a monkey-patching solution)

[schoeller]

@gswifort I have set up venv with launch.json as below:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Remote Attach",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "127.0.0.1",
            "justMyCode": false,
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "."
                }
            ],
            "env": {
                "PYTHONPATH": "$PYTHONPATH:C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs" 
            },
        }
    ]
}

VS Code states that env property is not allowed. I have tried adding PYTHONPATH="C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs" to the .env file. I still receive below error

Any hints?

Best regards

Sebastian

[gswifort]

I have the following debugging settings:

launch.json

{
    "name": "Autocad attach",
    "type": "debugpy",
    "request": "attach",
    "connect": {
        "host": "127.0.0.1",
        "port": 5678
    },
    "justMyCode": false
}

settings.json

{
    "python.envFile": "${workspaceFolder}/.env"
}

.env

PYTHONPATH = c:\ProgramData\Autodesk\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs

However, the error you show will not be resolved by setting the PYTHONPATH to the "stubs" folder. The "stubs" folder contains only *.pyi interfaces and Pyxx modules (containing the source code) are only available in the AutoCAD environment.

[CEXT-Dan]

Not quite sure how to deal with that vscode warning. The problem is PyLance can’t see python embedded inside C++ Those modules PyRx, PyGe etc.. are inside the .ARX file I’ve semi-resolved the issue by hiding it, see issue #3

[gswifort]

If the problem is only the display of errors, you can simply ignore them, as follows:

import PyRx as Rx  # type:ignore
import PyGe as Ge  # type:ignore
import PyGi as Gi  # type:ignore
import PyDb as Db  # type:ignore
import PyAp as Ap  # type:ignore
import PyEd as Ed  # type:ignore

If you use isort then also:

import PyRx as Rx  # isort:skip # type:ignore
import PyGe as Ge  # isort:skip # type:ignore
import PyGi as Gi  # isort:skip # type:ignore
import PyDb as Db  # isort:skip # type:ignore
import PyAp as Ap  # isort:skip # type:ignore
import PyEd as Ed  # isort:skip # type:ignore

[schoeller]

Thanks for the work. My current environment foresees the following working steps

1#Start BCAD and VS Code in parallel 2#Load script in BCAD manually via _PYLOAD 3#Start pydebug and call the remote debugger in VS Code 4#Alter and save script and restart from 2# via _PYLOAD (hopefully not killing BCAD)

Thus all modules must be installed accesible to the interpreter in BCAD (which is system-wide Python). VDEV is only available to VS Code. Is there any chance that I may have missed something and may directly launch and debug from within VS Code or reorganize my Python installation?

Best

Sebastian

[gswifort]

For BricsCad to see the virtual environment (installed packages), the directory <virtualenv>\Lib\site-packages must be added to sys.path To do this you have two options:

1. Set PYTHONPATH:

   # <virtualenv>\Scripts\activate.bat
   ...
   set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;%PYTHONPATH%

   and run BricsCad from the console with the virtual environment activated (for Autocad it is acad, for BricsCad probably bcad). For me, when I run Autocad from a virtual environment, the embedded Python does not see the PyRxStubs directory, so it also needs to be added to PYTHONPATH (You probably won't need this if you have the global PYTHONPATH variable set correctly - it is set during installation):

   # <virtualenv>\Scripts\activate.bat
   ...
   set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;C:\ProgramData\Bricscad\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs;%PYTHONPATH% 
   # check if the path is correct, I only replaced Autocad → Bricscad.

2. Add a path from the BCAD console:

   >>> PYCMDPROMPT
   >>> import sys
   >>> sys.path.insert(0, ".\<virtualenv>\Lib\site-packages")

[CEXT-Dan]

Some ideas:

-create loader modules for each host application. -The loader module will replace registry entries by adding env paths at runtime. -The loader module will be able to read a human editable configuration file so users can customize it.

-the installer will install main bulk of the package, i.e. wrappers, stubs, samples Users*username*\AppData\Local\Programs\PyRx

-Only the AutoCAD loader modules and PackageContents.xml will be installed in ApplicationPlugins -The loader modules for the clones will be installed in the PyRx folder where they can be added to a startup suite

• or maybe this can be a separate thing

[gswifort]

What do you mean by loader module?

[CEXT-Dan]

Another .ARX module that loads into Autocad, I'll need to hook into Autocads DLL paths and add the necessary paths

[CEXT-Dan]

Hi, Can you test if v1.3.002 is enough to solve this task?

I’ve added these functions: PyPreConfig_InitPythonConfig PyConfig_InitPythonConfig

But I don’t quite understand how they are used, but the goal is to be able to add configurations to the INI, that I can pass along. But I don’t know which ones you might need

[gswifort]

Hi, Wednesday at the earliest.

[gswifort]

Hi, I don't know how to use what you did. From what I see, I can use PyRx.ini, but only globally, this file is not searched in the current directory (virtual environment). For the virtual environment to work properly, it is necessary to adopt a different installation path scheme, as python.exe does in the virtual environment (python3x.dll does not set paths in the same way as python.exe, it has to be done manually). issue22213 and PEP 587 should be helpful.

[CEXT-Dan]

Yeah, It clear I don’t understand how all this works.

“From what I see, I can use PyRx.ini, but only globally, this file is not searched in the current directory (virtual environment).”

It’s not meant to be, the design is to tell AutoCAD the path to the DLLs. And to pass information to Py_PreInitialize and Py_InitializeFromConfig, I just don’t have a clue what parameters are needed

How about we start with: PYTHONISOLATED = 1 PYTHONEXECUTABLE = C:\path\to\venv\Scripts\python.exe

I’ll pass these to Py_InitializeFromConfig

[gswifort]

Yes, it is possible that this will be enough. However, you should probably pay attention to one more thing - which python3x.dll file to use. When creating a virtual environment, a pyvenv.cfg file is created in the %VIRTUAL_ENV% directory, containing, among others:

home = C:\Users\gswi\AppData\Local\Programs\Python\Python312

use the file located in this directory (we assume that the user may have different versions of python installed).

[schoeller]

@gswifort is this design platform-independant?

[gswifort]

I don't have access to macOS to test, but it seems so. PEP 587

[gswifort]

Analyzing the python venv module also shows that it is platform-independent

[gswifort]

And to determine whether to use an "isolated" configuration, I would check if the VIRTUAL_ENV environment variable exists.

[CEXT-Dan]

I’m preparing a build now; my thought is you must explicitly set PYTHONISOLATED in the INI I don’t want this to automatic, at least not yet

[gswifort]

I installed the latest version (1.3.006), but I don't know how to configure the .ini file. I tested in various ways, but it does not see the virtual environment. This is what the python.exe configuration looks like in the virtual environment:

(venv) K:\Programy\demos\pyrx_demo>python -m sysconfig
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "venv"

Paths:
        data = "K:\Programy\demos\pyrx_demo\venv"
        include = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        platinclude = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        platlib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
        platstdlib = "K:\Programy\demos\pyrx_demo\venv\Lib"
        purelib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
        scripts = "K:\Programy\demos\pyrx_demo\venv\Scripts"
        stdlib = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"

Variables:
        BINDIR = "\\pc-135\Users\GSWI\Documents\Programy\demos\pyrx_demo\venv\Scripts"
        BINLIBDEST = "K:\Programy\demos\pyrx_demo\venv\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp312-win_amd64.pyd"
        INCLUDEPY = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        LIBDEST = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"
        TZPATH = ""
        VERSION = "312"
        VPATH = "..\.."
        abiflags = ""
        base = "K:\Programy\demos\pyrx_demo\venv"
        exec_prefix = "K:\Programy\demos\pyrx_demo\venv"
        installed_base = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        installed_platbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        platbase = "K:\Programy\demos\pyrx_demo\venv"
        platlibdir = "DLLs"
        prefix = "K:\Programy\demos\pyrx_demo\venv"
        projectbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        py_version = "3.12.2"
        py_version_nodot = "312"
        py_version_nodot_plat = "312"
        py_version_short = "3.12"
        srcdir = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        userbase = "C:\Users\gswi\AppData\Roaming\Python"

and like this in autocad:

>>>: import sysconfig
>>>: sysconfig._main()
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "nt"

Paths:
 data = "c:\users\gswi\appdata\local\programs\python\python312"
 include = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 platinclude = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 platlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
 platstdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 purelib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
 scripts = "c:\users\gswi\appdata\local\programs\python\python312\Scripts"
 stdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"

Variables:
 BINDIR = "C:\Program Files\Autodesk\AutoCAD 2022"
 BINLIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 EXE = ".exe"
 EXT_SUFFIX = ".cp312-win_amd64.pyd"
 INCLUDEPY = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 LIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 TZPATH = ""
 VERSION = "312"
 VPATH = "..\.."
 abiflags = ""
 base = "c:\users\gswi\appdata\local\programs\python\python312"
 exec_prefix = "c:\users\gswi\appdata\local\programs\python\python312"
 installed_base = "c:\users\gswi\appdata\local\programs\python\python312"
 installed_platbase = "c:\users\gswi\appdata\local\programs\python\python312"
 platbase = "c:\users\gswi\appdata\local\programs\python\python312"
 platlibdir = "DLLs"
 prefix = "c:\users\gswi\appdata\local\programs\python\python312"
 projectbase = "C:\Program Files\Autodesk\AutoCAD 2022"
 py_version = "3.12.2"
 py_version_nodot = "312"
 py_version_nodot_plat = "312"
 py_version_short = "3.12"
 srcdir = "C:\Program Files\Autodesk\AutoCAD 2022"
 userbase = "C:\Users\gswi\AppData\Roaming\Python"

[CEXT-Dan]

Just of thought Instead of using the pycomand, try using the pyrx_onload.py in the bin directory

it loads early, so you might have to use a command or

import sysconfig

def OnPyLoadDwg(): print(sysconfig._main())

[CEXT-Dan]

I'll try to set one up over the next week and try

[gswifort]

Calling from the command level changes nothing.

The expected behavior is:

• create and activate a virtual environment:

  > python -m venv venv
  > venv\Scripts\activate.bat

• run AutoCAD from the virtual environment:

  > acad

• load the module:

  
  import PyRx as Rx # isort:skip
  import PyGe as Ge # isort:skip
  import PyGi as Gi # isort:skip
  import PyDb as Db # isort:skip
  import PyAp as Ap # isort:skip
  import PyEd as Ed # isort:skip

def PyRxCmd_doit(): import sys print(*sys.path, sep="\n")

you should get:

...\AppData\Local\Programs\Python\Python312\python312.zip ...\AppData\Local\Programs\Python\Python312\DLLs ...\AppData\Local\Programs\Python\Python312\Lib ...\AppData\Local\Programs\Python\Python312 ...\venv # !!! ...\venv\Lib\site-packages # !!!

Note that one `.ini` file is not a good solution because we want to have different directories in `sys.path` depending on the environment we run autocad from.

[CEXT-Dan]

I setup a venv

[PYRXSETTINGS] PYTHONINSTALLEDPATH =c:\users\dan\appdata\local\programs\python\python312 WXPYTHONPATH = M:\myenv\Lib\site-packages\wx PYRXSTUBPATH =C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs PYTHONISOLATED=1 PYTHONEXECUTABLE = M:\myenv\

output is

Regenerating model. Platform: "win-amd64" Python version: "3.12" Current installation scheme: "nt" Paths: data = "c:\users\dan\appdata\local\programs\python\python312" include = "c:\users\dan\appdata\local\programs\python\python312\Include" platinclude = "c:\users\dan\appdata\local\programs\python\python312\Include" platlib = "c:\users\dan\appdata\local\programs\python\python312\Lib\site-packages" platstdlib = "c:\users\dan\appdata\local\programs\python\python312\Lib" purelib = "c:\users\dan\appdata\local\programs\python\python312\Lib\site-packages" scripts = "c:\users\dan\appdata\local\programs\python\python312\Scripts" stdlib = "c:\users\dan\appdata\local\programs\python\python312\Lib" Variables: BINDIR = "M:\" BINLIBDEST = "c:\users\dan\appdata\local\programs\python\python312\Lib" EXE = ".exe" EXT_SUFFIX = ".cp312-win_amd64.pyd" INCLUDEPY = "c:\users\dan\appdata\local\programs\python\python312\Include" LIBDEST = "c:\users\dan\appdata\local\programs\python\python312\Lib" TZPATH = "" VERSION = "312" VPATH = "...." abiflags = "" base = "c:\users\dan\appdata\local\programs\python\python312" exec_prefix = "c:\users\dan\appdata\local\programs\python\python312" installed_base = "c:\users\dan\appdata\local\programs\python\python312" installed_platbase = "c:\users\dan\appdata\local\programs\python\python312" platbase = "c:\users\dan\appdata\local\programs\python\python312" platlibdir = "DLLs" prefix = "c:\users\dan\appdata\local\programs\python\python312" projectbase = "M:\" py_version = "3.12.2" py_version_nodot = "312" py_version_nodot_plat = "312" py_version_short = "3.12" srcdir = "M:\"

[gswifort]

Yes, I had the same effect. I showed sysconfig to illustrate where the differences are, what is more important is what paths we have in sys.path.

[CEXT-Dan]

Ok, I’ll goof around with it, over the next week.

Note that one .ini file is not a good solution

What’s your suggestion? We must know Python DLL path and wxPython dll path before loading otherwise AutoCAD will spit it out This is not going to be something that can change at runtime

[gswifort]

If you want to create a separate ini file, you can look for it in the directory where you run AutoCAD, but I recommend using the pyvenv.cfg with home = C:\Users\ gswi\AppData\Local\Programs\Python\Python312

[CEXT-Dan]

I had a validation error in my code, this seems correct now

[PYRXSETTINGS] PYTHONINSTALLEDPATH =c:\users\dan\appdata\local\programs\python\python312 WXPYTHONPATH =C:\path\to\myenv\Lib\site-packages\wx PYRXSTUBPATH =C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs PYTHONISOLATED=1 PYTHONEXECUTABLE = C:\path\to\myenv\Scripts\python.exe

PyRx version <1.3.006.20240805> loaded: Python Interpreter Loaded successfully! sys.executable=C:\path\to\myenv\Scripts\python.exe sys.prefix=C:\path\to\myenv sys.exec_prefix=C:\path\to\myenv sys.base_prefix=c:\users\dan\appdata\local\programs\python\python312 sys.base_exec_prefix=c:\users\dan\appdata\local\programs\python\python312 sys.path=c:\users\dan\appdata\local\programs\python\python312\python312.zip sys.path=c:\users\dan\appdata\local\programs\python\python312\DLLs sys.path=c:\users\dan\appdata\local\programs\python\python312\Lib sys.path=C:\Users\Dan\AppData\Local\Programs\Python\Python312 sys.path=C:\path\to\myenv sys.path=C:\path\to\myenv\Lib\site-packages sys.path=C:\Users\Dan\AppData\Local\Programs\PyRx\Stubs sys.path=c:\users\dan\appdata\local\programs\pyrx\bin\

[CEXT-Dan]

I had considered how to resolve the path to the .INI.

The search order for RxLoader25.0.arx to resolve PyRx25.0.arx is install dir then local. Resolution of PyRx.INI and pyrx_onload.py are always local to PyRx25.0.arx

I want to be able to switch environments easily for me to work in debug mode If you need different runtime configurations, you can copy those four files and put them where you like, and modify as you wish

[CEXT-Dan]

I don’t quite understand this

home = C:\Users\ gswi\AppData\Local\Programs\Python\Python312

In my mind, I can’t resolve ‘home’ without loading python, I can’t load python without knowing the exact path to wxPython

[gswifort]

Why can't you resolve "home" without loading python?

[CEXT-Dan]

Hi, It’s not clear to me where HOME originates from, do you mean in the windows environment? If it’s in the windows environment, I can resolve it.

[gswifort]

home is located in the pyvenv.cfg file generated by python when creating the virtual environment.

python -m venv venv_dir

When the virtual environment is activated, the VIRTUAL_ENV environment variable is created.

project/
└── venv_dir/ (%VIRTUAL_ENV%)
    ├── pyvenv.cfg
    └── ...

so you should load python from %home%\python312.dll

[CEXT-Dan]

Still confused, how do I find pyvenv.cfg? Pretend your starting form a clean system, all you have is the path to python in PATH?

[gswifort]

You can read the VIRTUAL_ENV environment variable.

[CEXT-Dan]

this is in the windows environment?

[gswifort]

Yes

[CEXT-Dan]

I went to my virtual environment folder and activated it I type in $Env:VIRTUAL_ENV and it’s empty

is this the right procedure?

[gswifort]

The question is what does it mean that you activated them... :) It is best to activate from the cmd console venv_dir\Scripts\activate.bat

[CEXT-Dan]

I see, the batch sets it in that instance of PowerShell. However it appears to be local, I’ll have to actually test if AutoCAD sees it

[gswifort]

If you run AutoCAD from the console (type acad in the console), it will see it.

[CEXT-Dan]

Acad didn’t launch for me, but it sounds reasonable that would be the case. I will try to resolve VIRTUAL_ENV first, then fall back to the .INI for now

[CEXT-Dan]

it sees it

Customization file loaded successfully. Customization Group: FEATUREDAPPS Regenerating model. true-C:\path\to\myenv AutoCAD menu utilities loaded.Cancel