Can not import from compiled DLL

[htlcnn]

I have a python file with content:

# D:\HTL\Desktop\test\htladdin.py
print("printed from inside dll")

I compiled it to htladdin.dll with this command:

"C:\Program Files (x86)\IronPython 2.7\ipy64.exe" "C:\Program Files (x86)\IronPython 2.7\Tools\Scripts\pyc.py" "D:\HTL\Desktop\test\htladdin.py" /out:htladdin

Then I add reference to it from my pyrevit script:

# E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl.extension\HTL.tab\Test.panel\Test.pushbutton\script.py
# -*- coding: utf-8 -*-

import clr
import sys

sys.path.append(r'D:\HTL\Desktop\test')
clr.AddReferenceToFile('htladdin.dll')

import htladdin
print(htladdin)

When I run the script inside Revit, Exception raised:

IronPython Traceback:
Traceback (most recent call last):
 File "E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl.extension\HTL.tab\Test.panel\Test.pushbutton\script.py", line 9, in 
ImportError: No module named htladdin

Script Executor Traceback:
IronPython.Runtime.Exceptions.ImportException: No module named htladdin
 at Microsoft.Scripting.Runtime.LightExceptions.ThrowException(LightException lightEx)
 at Microsoft.Scripting.Runtime.LightExceptions.CheckAndThrow(Object value)
 at Microsoft.Scripting.Interpreter.FuncCallInstruction`2.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
 at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
 at Microsoft.Scripting.Hosting.ScriptSource.Execute(ScriptScope scope)
 at PyRevitBaseClasses.ScriptExecutor.ExecuteScript(String sourcePath, String syspaths, String cmdName, String cmdUniqueName, Boolean forcedDebugMode, Boolean altScriptMode, Dictionary`2& resultDict)

I was succeeded in adding reference and import the compiled DLL from ironpython console:

PS D:\HTL\Desktop\test> & 'C:\Program Files (x86)\IronPython 2.7\ipy64.exe'
IronPython 2.7.3 (2.7.0.40) on .NET 4.0.30319.42000 (64-bit)
Type "help", "copyright", "credits" or "license" for more information.
>>> import clr
>>> import sys
>>> sys.path.append(r'D:\HTL\Desktop\test')
>>> clr.AddReferenceToFile('htladdin.dll')
>>> import htladdin
printed from inside dll
>>> print(htladdin)
<module 'htladdin' from 'htladdin'>
>>>

[eirannejad]

use clr.AddReferenceToFileAndPath and provide absolute path to the dll.

[htlcnn]

@eirannejad still same Exception:

# -*- coding: utf-8 -*-

import clr
import sys

clr.AddReferenceToFileAndPath(r'D:\HTL\Desktop\test\htladdin.dll')

import htladdin
print(htladdin)

IronPython Traceback:
Traceback (most recent call last):
 File "E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl.extension\HTL.tab\Test.panel\Test.pushbutton\script.py", line 9, in 
ImportError: No module named htladdin

Script Executor Traceback:
IronPython.Runtime.Exceptions.ImportException: No module named htladdin
 at Microsoft.Scripting.Runtime.LightExceptions.ThrowException(LightException lightEx)
 at Microsoft.Scripting.Runtime.LightExceptions.CheckAndThrow(Object value)
 at Microsoft.Scripting.Interpreter.FuncCallInstruction`2.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
 at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
 at Microsoft.Scripting.Hosting.ScriptSource.Execute(ScriptScope scope)
 at PyRevitBaseClasses.ScriptExecutor.ExecuteScript(String sourcePath, String syspaths, String cmdName, String cmdUniqueName, Boolean forcedDebugMode, Boolean altScriptMode, Dictionary`2& resultDict)

[eirannejad]

I believe in your IronPython tests you're importing the python file not the DLL file. You have the python file and the dll file in the same directory and add that to the sys.path

In your pyRevit script you're directly linking to the dll and it won't be able to import.

I tested my IronPython (same version as yours) and fails the import.

IronPython 2.7.3 (2.7.0.40) on .NET 4.0.30319.42000 (64-bit)
Type "help", "copyright", "credits" or "license" for more information.
>>> import clr
>>> clr.AddReferenceToFileAndPath(r'C:\Users\eirannejad\Desktop\htladdin.dll')
>>> import htladdin
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named htladdin
>>>

IronPython 2.7.3 (2.7.0.40) on .NET 4.0.30319.42000 (64-bit)
Type "help", "copyright", "credits" or "license" for more information.
>>> import clr
>>> import sys
>>> sys.path.append(r'C:\Users\eirannejad\Desktop')
>>> clr.AddReference('htladdin')
>>> import htladdin
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named htladdin
>>>

This is my script and the command to compile it:

def run(value):
    print("printed from inside dll {}".format(value))

"C:\Program Files (x86)\IronPython 2.7\ipy64.exe"\
"C:\Program Files (x86)\IronPython 2.7\Tools\Scripts\pyc.py"\
/main:"C:\Users\eirannejad\Desktop\htladdin.py"
/out:htladdin
/target:dll
/platform:x64
/embed

I'm closing this since this is not a pyRevit issue. Keep the conversation going please, I'd love to know the solution to this.

My hunch is that the compiled DLL does not have a namespace and it throws IronPython off. See the dotPeek view of the dll.

If you try import DLRCachedCode it works since it's a type inside that dll.

I think you gotta find a way to compile the python into a dll that has a namespace other than the root namespace.

[htlcnn]

@eirannejad Actually I renamed htladdin.py to htladdin1.py before adding reference to the dll file, so I'm sure I was referencing and import dll file. To clarify, I've now moved the python file to \py folder, then copied dll file outside to add reference.

This is my compile command:

"C:\Program Files (x86)\IronPython 2.7\ipy64.exe" "C:\Program Files (x86)\IronPython 2.7\Tools\Scripts\pyc.py" "D:\HTL\Desktop\test\py\htladdin.py" /target:dll

Note that I didn't add /main before file name because I read in pyc.py:

EXE/WinEXE specific options:
    /main:main_file.py                        Main file of the project (module to be executed first)
    /platform:x86                             Compile for x86 only
    /platform:x64                             Compile for x64 only
    /embed                                    Embeds the generated DLL as a resource into the executable which is loaded at runtime
    /standalone                               Embeds the IronPython assemblies into the stub executable.
    /mta                                      Set MTAThreadAttribute on Main instead of STAThreadAttribute, only valid for /target:winexe

(/target:dll is default so I think it's optional in this case)

With that compile command, I was successful in adding reference to dll file, despite the content of the htladdin.py file:

Compiling message:
Input Files:
    D:\HTL\Desktop\test\py\htladdin.py
Output:
    htladdin
Target:
    Dll
Platform:
    ILOnly, PE32Plus
Machine:
    AMD64

Compiling...
Saved to htladdin
[Finished in 0.4s]

PS D:\HTL\Desktop\test> ls

    Directory: D:\HTL\Desktop\test

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       25/11/2017  11:28 AM                py
-a----       25/11/2017  11:39 AM           4608 htladdin.dll

PS D:\HTL\Desktop\test> & 'C:\Program Files (x86)\IronPython 2.7\ipy64.exe'
IronPython 2.7.3 (2.7.0.40) on .NET 4.0.30319.42000 (64-bit)
Type "help", "copyright", "credits" or "license" for more information.
>>> import clr
>>> clr.AddReferenceToFileAndPath(r'D:\HTL\Desktop\test\htladdin.dll')
>>> import htladdin
printed from inside dll
>>>

[htlcnn]

@eirannejad Digging a little bit further, I noticed that I had successfully added reference to the dll file. This is my script.py code:

# -*- coding: utf-8 -*-
import clr
import sys

clr.AddReferenceToFileAndPath(r'D:\HTL\Desktop\test\htladdin.dll')
print(clr.References)

and the output:

(<mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089>, 
<System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089>, 
<RevitAPI, Version=18.0.0.0, Culture=neutral, PublicKeyToken=null>, 
<RevitAPIUI, Version=18.0.0.0, Culture=neutral, PublicKeyToken=null>, 
<pyRevit_2018_a567b41e08226f67_PyRevitBaseClasses, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null>, 
<htladdin, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null>) 

[eirannejad]

Yeah the DLL gets loaded but the clr.AddReference. It's when you want to import that it can not find the htladdin namespace inside the DLL

[htlcnn]

@eirannejad I've read and tried to use clr.CompileModules to make .dll file instead of using pyc.py. The compiled dll was successfully imported into ironpython interpreter (I renamed the original .py module to make sure it imports from dll file). Here's the article's link: http://www.voidspace.org.uk/ironpython/dark-corners.shtml#static-compilation-of-python-code But I still can't figure out why the same dll couldn't be imported into pyrevit scripts

Another reference from MSDN Blog: https://blogs.msdn.microsoft.com/srivatsn/2008/08/06/static-compilation-of-ironpython-scripts/

[eirannejad]

@htlcnn I'm pretty sure the generated dll is created using IronPython 2.7.3 and works with that version only. Go to pyRevit settings and activate the Dynamo Compatibility mode, save and reload Revit, it will load pyRevit using the 2.7.3 engine.

I just tested this and it works. Here are the 3 scripts:

Source module to be compiled: (file name ehsantest.py)

def somefunc(someval):
    print(someval)
somefunc(15)

Compiler script:

import sys
import os.path as op
thispath = op.dirname(__file__)
source = op.join(thispath, 'ehsantest.py')
dest = op.join(thispath, 'compiled', 'compiledehsantest.dll')
import clr
clr.CompileModules(dest, source)

$ ipy64 compile.py

pyRevit script:

import clr
import sys

# add the path to the dll directory
sys.path.append(r"C:\Users\eirannejad\Desktop\compiled")
clr.AddReference("compiledehsantest")

# original python file name is the namespace
import ehsantest

I'll test compiling in 2.7.7 and testing with 2.7.7 engine in pyRevit

[eirannejad]

Confirmed. Install IronPython 2.7.7 and compile using that. Then it perfectly loads in pyRevit. I'll do a blog post on this.

[htlcnn]

@eirannejad sorry for digging this up. I read about Bundle Lib & bin, it says I can put some dll files in my extension's bin folder. For example: htl.extension\bin\license.dll. Then I can AddReference and import it for use in other child bundles. Here's my child bundle's code:

# -*- coding: utf-8 -*-
import clr
clr.AddReference('license.dll')
for r in clr.References:
    if str(r.GetName()).startswith('license'):
        print(r.CodeBase)

import license

Dll file is referenced successfully, but I can not import the namespace from it. Here is output:

file:///.../htl.extension/bin/license.DLL

IronPython Traceback:
Traceback (most recent call last):
 File "...\htl.extension\HTL.tab\Test.panel\views.stack\Test1.pushbutton\script.py", line 8, in <module>
ImportError: No module named license

Script Executor Traceback:
IronPython.Runtime.Exceptions.ImportException: No module named license
 at Microsoft.Scripting.Runtime.LightExceptions.ThrowException(LightException lightEx)
 at Microsoft.Scripting.Runtime.LightExceptions.CheckAndThrow(Object value)
 at Microsoft.Scripting.Interpreter.FuncCallInstruction`2.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
 at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
 at PyRevitLabs.PyRevit.Runtime.IronPythonEngine.Execute(ScriptRuntime& runtime)

Here's how I compiled the script:

PS ...\ironpython\IronPython-2.7.7> .\ipy64.exe
IronPython 2.7.7 (2.7.7.0) on .NET 4.0.30319.42000 (64-bit)
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import os
>>> src = r"...\htl.extension\lib\htl\license.py"
>>> dst = r"...\htl.extension\lib\htl\license.dll"
>>> import clr
>>> clr.CompileModules(dst, src)
>>> sys.path.append(r"...\htl.extension\lib\htl\dll")
>>> clr.AddReference('license')
>>> import license
>>> dir(license)
['Cryptography', 'Directory', 'File', 'InvalidLicense', 'LicFileNotFoundException', 'ManagementObjectSearcher', 'MessageBox', 'MessageBoxButtons', 'MessageBoxIcon', 'NoLicenseInfoException', 'Path', 'Regex', 'System', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_generate_activation_code', '_generate_request_code', 'check_license', 'clr', 'generate_md5', 'get_hdd_serial_number', 'get_lic_path', 'get_license_info', 'json', 'save_license', 'sys']

Then I copied the compiled dll to extension\bin folder.

Here's my pyRevit env:

PS E:\Setup\coding\ironpython\IronPython-2.7.7> pyrevit env
==> Registered Clones (full git repos)
==> Registered Clones (deployed from archive/image)
master | Deploy: "basepublic" | Branch: "master" | Version: "4.7-beta4" | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
==> Attachments
master | Product: "2020.1 Update" | Engine: 277 | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master" | Manifest: "C:\Users\HTL\AppData\Roaming\Autodesk\Revit\Addins\2020\pyRevit.addin"
master | Product: "2018.3.1" | Engine: 277 | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master" | Manifest: "C:\Users\HTL\AppData\Roaming\Autodesk\Revit\Addins\2018\pyRevit.addin"
==> Installed Extensions
htl | Type: Unknown | Repo: "git@gitlab.com:hoangthanhlong/pyrevitscripts.git" | Installed: "...\htl.extension"
pyRevitDevTools | Type: Unknown | Repo: "" | Installed: "...\pyRevitDevTools.extension"
==> Default Extension Search Path
C:\Users\HTL\AppData\Roaming\pyRevit\Extensions
==> Extension Search Paths
...\pyRevit extensions
...\pyRevit extensions DEV
==> Extension Sources - Default
https://github.com/eirannejad/pyRevit/raw/master/extensions/extensions.json
==> Extension Sources - Additional
==> Installed Revits
2020.1 Update | Version: 20.1.0.81 | Build: 20190725_1135(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2020"
2018.3.1 | Version: 18.3.1.2 | Build: 20180423_1000(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2018"
==> Running Revit Instances
PID: 17424 | 2018.3.1 | Version: 18.3.1.2 | Build: 20180423_1000(x64) | Language: 0 | Path: "C:\Program Files\Autodesk\Revit 2018"
==> User Environment
Microsoft Windows 10 [Version 10.0.18362]
Executing User: HTL\HTL
Active User: HTL\HTL
Admin Access: No
%APPDATA%: "C:\Users\HTL\AppData\Roaming"
Latest Installed .Net Framework: 4.8
Installed .Net Target Packs: v3.5 v4.0 v4.5 v4.5.1 v4.5.2 v4.6 v4.6.1 v4.6.2 v4.7 v4.7.1 v4.7.2 v4.8 v4.X
Installed .Net-Core Target Packs: v2.1.602
pyRevit CLI 0.16.0.0
PS E:\Setup\coding\ironpython\IronPython-2.7.7>

Did I miss something?

[htlcnn]

@eirannejad It's strange that although I compiled the dll using IronPython 2.7.7, I had to switch to IronPython 2.7.3 Engine in pyRevit settings in order to import and use the dll. Is there a mis-config here?

C:\Users\HTL>pyrevit env
==> Registered Clones (full git repos)
dev | Branch: "develop" | Version: "4.7.4:fb1ea7f" | Path: "C:\pyRevit\dev\dev"
==> Registered Clones (deployed from archive/image)
master | Deploy: "basepublic" | Branch: "master" | Version: "4.7-beta2" | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
==> Attachments
dev | Product: "2018.3.1" | Engine: 273 | Path: "C:\pyRevit\dev\dev" | Manifest: "C:\Users\HTL\AppData\Roaming\Autodesk\Revit\Addins\2018\pyRevit.addi
n"
==> Installed Extensions
htl | Type: Unknown | Repo: "git@gitlab.com:hoangthanhlong/pyrevitscripts.git" | Installed: "E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl.extens
ion"
==> Default Extension Search Path
C:\Users\HTL\AppData\Roaming\pyRevit\Extensions
==> Extension Search Paths
E:\Setup\UCE\Autodesk\Revit\pyRevit extensions
==> Extension Sources - Default
https://github.com/eirannejad/pyRevit/raw/master/extensions/extensions.json
==> Extension Sources - Additional
==> Installed Revits
2018.3.1 | Version: 18.3.1.2 | Build: 20180423_1000(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2018"
==> Running Revit Instances
==> User Environment
Microsoft Windows 7 [Version 6.1.7601]
Executing User: HTL-PC\HTL
Active User: HTL-PC\HTL
Adming Access: Yes
%APPDATA%: "C:\Users\HTL\AppData\Roaming"
Latest Installed .Net Framework: 4.8
Installed .Net Target Packs: v3.5 v4.0 v4.5 v4.5.1 v4.5.2 v4.6 v4.6.1 v4.7 v4.7.1 v4.7.2 v4.X
Installed .Net-Core Target Packs: v2.2.203 v3.0.100-preview8-013656
pyRevit CLI 0.14.0.0

[eirannejad]

Huh I don't think so. I have a test tool for compiling and importing DLLs in the pyRevitDev extension and it passes the compile and import test under IronPython 2.7.7

[htlcnn]

@eirannejad I ran your test on IronPython engine 2.7.3, it compiled and imported dll successfully. When I switched to IronPython engine 2.7.7, it compiled successfully but it imported .py file instead of .dll file. I added a line to the code to show imported path:

import ipycompiletest
print(ipycompiletest.__file__)

Output on IronPython engine 2.7.3:

Compiled module level code works.
ipycompiletest
Compiled function works.
Compiled type works.

Output on IronPython engine 2.7.7:

Compiled module level code works.
C:\pyRevit\dev\dev\extensions\pyRevitDevTools.extension\pyRevitDev.tab\Debug.panel\Engine Tests.pulldown\Test IronPython Compile.pushbutton\ipycompiletest.py
Compiled function works.
Compiled type works.

---

On 2.7.7, if I modified the code to rename .py file to other name, it failed to import.

import sys
import os.path as op
import os
import clr

from pyrevit import USER_SYS_TEMP
from pyrevit import script
from pyrevit.framework import IO

# compile
try:
    source = script.get_bundle_file('ipycompiletest.py')
    dest = op.join(USER_SYS_TEMP, 'compiledipytest.dll')
    clr.CompileModules(dest, source)
    new_src = source.replace('ipycompiletest', 'ipycompiletest_old')
    os.rename(source, new_src)
except IO.IOException as ioerr:
    print('DLL file already exists...')
except Exception as cerr:
    print('Compilation failed: {}'.format(cerr))

# import test
sys.path.append(USER_SYS_TEMP)
clr.AddReferenceToFileAndPath(dest)

import ipycompiletest
print(ipycompiletest.__file__)
ipycompiletest.compile_test('Compiled function works.')

ipycompiletest.CompiledType('Compiled type works.')

Output:

IronPython Traceback:
Traceback (most recent call last):
 File "C:\pyRevit\dev\dev\extensions\pyRevitDevTools.extension\pyRevitDev.tab\Debug.panel\Engine Tests.pulldown\Test IronPython Compile.pushbutton\script.py", line 26, in <module>
ImportError: No module named ipycompiletest

Script Executor Traceback:
IronPython.Runtime.Exceptions.ImportException: No module named ipycompiletest
 at Microsoft.Scripting.Runtime.LightExceptions.ThrowException(LightException lightEx)
 at Microsoft.Scripting.Runtime.LightExceptions.CheckAndThrow(Object value)
 at Microsoft.Scripting.Interpreter.FuncCallInstruction`2.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
 at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
 at PyRevitLabs.PyRevit.Runtime.IronPythonEngine.Execute(ScriptRuntime& runtime)

By the way, is it comfortable to discuss on a closed issue? Should I open a new issue?

[digineer]

By coincidence I was also testing how to compile iron python dlls and than calling them from pyrevit scripts this morning. At first I had the same issue: It worked when compiling with IronPython 2.7.3 and selecting pyRevit Engine 2.7.3 and then it did not work when compiling with IronPython 2.7.7 and pyRevit Engine 2.7.7. I don't know what I did but than suddenly it worked with IronPython 2.7.7 compilation and pyRevit Engine 2.7.7. Could this be a CACHE issue? Maybe deleting the files in C:\Users\YOURUSERNAME\AppData\Roaming\pyRevit\2019 will help.

BTW: My end-goal is to use this workflow to create dlls from pyRevit scripts that can then be used with the Forge Revit Design Automation API which is now available to the public (see Autodesk Channel Newsletter from this morning). Writing pyRevit scripts and running them in the cloud. HOW COOL WOULD THAT BE!!!

[htlcnn]

@digineer did you rename/move the source .py file to another path that it wouldn't be imported into your script.py? If you didn't, I'm afraid .py file was imported instead of compiled .dll file.

[digineer]

@htlcnn Yes, I did. The source.py file lives on a completely different path and only the compiled dll is copied into the bundle from where it is loaded. To make extra sure I also deleted the original source.py and it still works.

[htlcnn]

@digineer what's your pyrevit env? I couldn't get it imported on 2.7.7. I wonder if there was a change in import order from 2.7.3 to 2.7.7.

[digineer]

@htlcnn I am running pyRevit 4.7.4 from the public installer.

[htlcnn]

@eirannejad Could you please explain the process of executing a button bundle? I tried to read your code but it seems to be quite complex for me because now pyRevit supports many different engines. I created a C# addin that creates an IronPython engine and run a .py script to import a compiled dll. It was successful. Most of the C# code to create IronPython engine was from pyRevit. Please have a look: https://github.com/htlcnn/RevitIronPythonEngineTest Many thanks!

[vasanth15]

@htlcnn I ran into the same issue, I am using ironPython 2.7.7, followed the same steps as mentioned above, successfully compiled the dlls, but hitting a road block here, for some reason, iron python can't find the module name. Were you able to import the dlls successfully? Thanks in advance.

[htlcnn]

@vasanth15 Try 2.7.11 engine

[vasanth15]

@htlcnn Yup, it works with 2710 version. Many thanks.!

With 2711, for some reason, on my pyrevit, it says it is a custom iron python engine (I hope, it is an interim version), it was able to identify the dll however, there were places where arguments mismatch occurred in pyrevit library.