Debugging pyRevit engines and IronPython Compile Test command

htlcnn commented 4 years ago

Describe the bug I had this issue when starting Revit 2020.2 with pyRevit 4.7.6 (IronPython 2.7.9 core) and did not load pyRevit. My Revit 2018.3 had no issue (no alert window) and loaded pyRevit normally.

Screenshot

To Reproduce

Before having this issue, still with Revit 2020.2 & IronPython 2.7.9, I had issue No module named _wpf, Revit did not load pyRevit either. Traceback said line 85 failed: https://github.com/eirannejad/pyRevit/blob/4319ff011c36f24b26b2394d6862e685e566f26b/pyrevitlib/pyrevit/framework.py#L81-L85 So I recalled discussing in this issue and made a small change to those lines in framework.py and the _sqlite3 issue appeared:

# I added 2 more comparison for IronPython 2.7.7 and IronPython 2.7.9
if compat.PY3 or compat.IRONPY277 or compat.IRONPY279: 
    import IronPython
    wpf = IronPython.Modules.Wpf
else:
    import wpf

I tried adding 'IronPython.SQLite' to this line, hoping it was loaded to solve the issue, but no luck.

Desktop :

OS: Windows 10
pyRevit Version 4.7.6

pyRevit Environment:

PS C:\Users\HTL\AppData\Roaming\pyRevit-Master\bin\engines\279> pyrevit env
==> Registered Clones (full git repos)
==> Registered Clones (deployed from archive/image)
master | Deploy: "basepublic" | Branch: "master" | Version: "4.7.6" | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
==> Attachments
master | Product: "Autodesk Revit 2020" | Engine: 279 | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master" | Manifest: "C:\Users\HTL\AppData\Roaming\Autodesk\Revit\Addins\2020\pyRevit.addin"
master | Product: "2018.3.1" | Engine: 279 | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master" | Manifest: "C:\Users\HTL\AppData\Roaming\Autodesk\Revit\Addins\2018\pyRevit.addin"
==> Installed Extensions
htl | Type: Unknown | Repo: "" | Installed: "E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl\htl.extension"
==> Default Extension Search Path
C:\Users\HTL\AppData\Roaming\pyRevit\Extensions
==> Extension Search Paths
E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl
==> Extension Sources - Default
https://github.com/eirannejad/pyRevit/raw/master/extensions/extensions.json
==> Extension Sources - Additional
==> Installed Revits
Autodesk Revit 2020 | Version: 20.2.12.1 | Build: 20200210_1400(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2020"
2018.3.1 | Version: 18.3.1.2 | Build: 20180423_1000(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2018"
==> Running Revit Instances
Error: Object reference not set to an instance of an object.
Run with "--debug" option to see debug messages

htlcnn commented 4 years ago

@eirannejad can you reproduce this?

eirannejad commented 4 years ago

I tested this and can not seem to replicate the issue. :/ Might be another one of those IronPython conflicts

eirannejad commented 4 years ago

@htlcnn Would you check to see which IronPython dll is loaded in memory when pyRevit is running? Use the "Find Referenced Assemblies" tool

htlcnn commented 4 years ago

@eirannejad Since pyRevit did not load with engine 2.7.9, I switched to other engines. It loaded successfully with 2.7.7, 2.7.8 and 2.7.10. For 2.7.7 and 2.7.10, IronPython Compile Test failed. It worked for 2.7.8. I ran Find Referenced Assemblies and see both IronPython 2.7.7 (or 2.7.8/2.7.10) loaded by pyRevit and IronPython 2.7.8 loaded by Dynamo. Is that the reason for the conflict? That was not the case for Revit 2018 bacause Dynamo 1.3 on Revit 2018 has its own DSIronPython.dll. However, IronPython Compile test on Revit 2018 succeeded for 2.7.3 and 2.7.9, but failed for 2.7.7, 2.7.8 and 2.7.10 engines. What should be done to avoid such conflicts?

This is result of Find Referenced Assemblies on my Revit 2020: https://pastebin.com/xJqjTM7V

htlcnn commented 4 years ago

More detailed information:

Revit Version	IronPython engine	pyRevit loaded	IronPython Compile Test command
2018.3	2.7.3	OK	OK
2018.3	2.7.7	OK	Fail
2018.3	2.7.8	OK	Fail
2018.3	2.7.9	OK	OK
2018.3	2.7.10	OK	Fail
2020.2	2.7.3	Fail	x
2020.2	2.7.7	OK	Fail
2020.2	2.7.8	OK	OK
2020.2	2.7.9	Fail	x
2020.2	2.7.10	OK	Fail

eirannejad commented 4 years ago

Uh this table is soooo helpful! Thank you

htlcnn commented 4 years ago

After deleting all *IronPython folders in C:\Windows\Microsoft.NET\assembly\GAC_MSIL\, reinstalling pyRevit 4.8.4 and restarting the PC: Revit Version	IronPython engine	pyRevit loaded	IronPython Compile Test command
2018.3	2.7.3	Fail (Image)	x
2018.3	2.7.7	OK	Fail
2018.3	2.7.8	OK	Fail
2018.3	2.7.9	Fail (Image)	x
2018.3	2.7.10	OK	Fail
2020.2	2.7.3	Fail (Image)	x
2020.2	2.7.7	OK	Fail
2020.2	2.7.8	OK	OK
2020.2	2.7.9	Fail (Image)	x
2020.2	2.7.10	OK	Fail

htlcnn commented 4 years ago

I was testing on another PC in my office that had both 2018.3 and 2020 running. Results were a bit different from mine, then I noticed that the other PC installed DynamoRevit2.0.4 so Dynamo version selector was loaded instead of Dynamo itself. There might be something related to how Revit 2018 handles Dynamo Results on that other PC: Revit Version	IronPython engine	pyRevit loaded	IronPython Compile Test command
2018.3	2.7.3	OK	OK
2018.3	2.7.7	OK	OK
2018.3	2.7.8	OK	OK
2018.3	2.7.9	OK	OK
2018.3	2.7.10	OK	OK
2020.2	2.7.3	Fail	x
2020.2	2.7.7	OK	Fail
2020.2	2.7.8	OK	OK
2020.2	2.7.9	OK	Fail
2020.2	2.7.10	OK	Fail

I was also messing things up my PC by deleting IronPython dll files in the GAC_MSIL folder, so I moved the backup dll files back to their original places and used gacutil to uninstall IronPython dlls. Some versions I installed manually before were uninstalled. There was version 2.7.0 that could not be uninstall because it was required by another program (gacutil said that, not sure what that program was). After uninstalling all manually installed IronPython versions in the GAC, I tested again with Dynamo2.0.4 installed on my PC and the results were the same with the other PC.

htlcnn commented 3 years ago

@eirannejad I'm running pyRevit 4.8.5, Revit 2018.3, IronPython 2.7.11 engine (as I found it not stable with several engines in the past, I decided I would try your custom 2.7.11 engine). Test IronPython Compile button fails and says it could not find pyRevitLabs.Microsoft.Dynamic 1.3.0.0 while I could see it in loaded assemblies (too long, should not paste here).

Here is error log:

IronPython Traceback:
Traceback (most recent call last):
 File "...pyRevit extensions\htl\htl.extension\HTL.tab\Dev.panel\archive1.stack\Archive.pulldown\Test IronPython Compile.pushbutton\script.py", line 21, in <module>
IOError: [Errno 2] Could not load file or assembly 'pyRevitLabs.Microsoft.Dynamic, Version=1.3.0.0, Culture=neutral, PublicKeyToken=7f709c5b713576e1' or one of its dependencies. The system cannot find the file specified.

Script Executor Traceback:
System.IO.FileNotFoundException: Could not load file or assembly 'pyRevitLabs.Microsoft.Dynamic, Version=1.3.0.0, Culture=neutral, PublicKeyToken=7f709c5b713576e1' or one of its dependencies. The system cannot find the file specified.
File name: 'pyRevitLabs.Microsoft.Dynamic, Version=1.3.0.0, Culture=neutral, PublicKeyToken=7f709c5b713576e1'
 at System.ModuleHandle.ResolveType(RuntimeModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
 at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
 at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
 at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(CustomAttributeRecord caRecord, MetadataImport scope, Assembly& lastAptcaOkAssembly, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, Object[] attributes, IList derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg)
 at System.Reflection.CustomAttribute.IsCustomAttributeDefined(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Int32 attributeCtorToken, Boolean mustBeInheritable)
 at System.Reflection.CustomAttribute.IsDefined(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
 at Microsoft.Scripting.SavableScriptCode.LoadFromAssembly(ScriptDomainManager runtime, Assembly assembly)
 at IronPython.Runtime.Binding.PythonBinder.LoadScriptCode(PythonContext pc, Assembly asm)
 at IronPython.Runtime.Binding.PythonBinder.DomainManager_AssemblyLoaded(Object sender, AssemblyLoadedEventArgs e)
 at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
 at Microsoft.Scripting.Runtime.ScriptDomainManager.LoadAssembly(Assembly assembly)
 at IronPython.Runtime.ClrModule.AddReferenceToFileAndPath(CodeContext context, String file)
 at IronPython.Runtime.ClrModule.AddReferenceToFileAndPath(CodeContext context, String[] files)
 at Microsoft.Scripting.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run4[T0,T1,T2,T3,TRet](T0 arg0, T1 arg1, T2 arg2, T3 arg3)
 at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
 at Microsoft.Scripting.Interpreter.DynamicInstruction`4.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
 at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
 at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
 at PyRevitLabs.PyRevit.Runtime.IronPythonEngine.Execute(ScriptRuntime& runtime)

=== Pre-bind state information ===
LOG: DisplayName = pyRevitLabs.Microsoft.Dynamic, Version=1.3.0.0, Culture=neutral, PublicKeyToken=7f709c5b713576e1
 (Fully-specified)
LOG: Appbase = file:///C:/Program Files/Autodesk/Revit 2018/
LOG: Initial PrivatePath = NULL
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Program Files\Autodesk\Revit 2018\Revit.exe.Config
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/pyRevitLabs.Microsoft.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/pyRevitLabs.Microsoft.Dynamic/pyRevitLabs.Microsoft.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/SDA/bin/pyRevitLabs.Microsoft.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/SDA/bin/pyRevitLabs.Microsoft.Dynamic/pyRevitLabs.Microsoft.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/pyRevitLabs.Microsoft.Dynamic.EXE.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/pyRevitLabs.Microsoft.Dynamic/pyRevitLabs.Microsoft.Dynamic.EXE.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/SDA/bin/pyRevitLabs.Microsoft.Dynamic.EXE.
LOG: Attempting download of new URL file:///C:/Program Files/Autodesk/Revit 2018/SDA/bin/pyRevitLabs.Microsoft.Dynamic/pyRevitLabs.Microsoft.Dynamic.EXE.

Source code of the script.py, mostly taken from your pyRevitDev extension:

import sys
import os.path as op
import clr

from pyrevit import USER_SYS_TEMP
from pyrevit import script
from pyrevit.framework import IO

# compile
try:
    source = script.get_bundle_file('f\\ipycompiletest.py')
    dest = op.join(USER_SYS_TEMP, 'coyrtompiledipytest.dll')
    clr.CompileModules(dest, source)
except IO.IOException as ioerr:
    print('DLL file already exists...')
except Exception as cerr:
    print('Compilation failed: {}'.format(cerr))

# import test
sys.path.append(USER_SYS_TEMP)
clr.AddReferenceToFileAndPath(dest)
import os
# os.rename(source, source.replace('ipycompiletest', 'ipycompiletest_old'))

import ipycompiletest

print(ipycompiletest)

ipycompiletest.compile_test('Compiled function works.')

ipycompiletest.CompiledType('Compiled type works.')

Line 21 is clr.AddReferenceToFileAndPath(dest).

To Reproduce Steps to reproduce the behavior:

Attach 2.7.11 engine
Run Test IronPython Compile button

Expected behavior Compiled script should be imported successfully

Desktop (please complete the following information): pyrevit env

PS C:\WINDOWS\system32> pyrevit env
==> Registered Clones (full git repos)
==> Registered Clones (deployed from archive/image)
master | Deploy: "basepublic" | Branch: "master" | Version: "4.8.5" | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
==> Attachments
master | Product: "2020.2.1 (New Install)" | Engine: DEFAULT (2711) | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
master | Product: "2018.3.1" | Engine: DEFAULT (2711) | Path: "C:\Users\HTL\AppData\Roaming\pyRevit-Master"
==> Installed Extensions
htl | Type: Unknown | Repo: "" | Installed: "E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl\htl.extension"
==> Default Extension Search Path
C:\Users\HTL\AppData\Roaming\pyRevit\Extensions
==> Extension Search Paths
E:\Setup\UCE\Autodesk\Revit\pyRevit extensions\htl
==> Extension Sources - Default
https://github.com/eirannejad/pyRevit/raw/master/extensions/extensions.json
==> Extension Sources - Additional
==> Installed Revits
2020.2.1 (New Install) | Version: 20.2.12.1 | Build: 20200210_1400(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2020\"
2018.3.1 | Version: 18.3.1.2 | Build: 20180423_1000(x64) | Language: 1033 | Path: "C:\Program Files\Autodesk\Revit 2018\"
==> Running Revit Instances
==> User Environment
Microsoft Windows 10 [Version 10.0.18363]
Executing User: HTL-PC\HTL
Active User: HTL-PC\HTL
Admin Access: Yes
%APPDATA%: "C:\Users\HTL\AppData\Roaming"
Latest Installed .Net Framework: 4.8
No .Net Target Packs are installed.
No .Ne-Core Target Packs are installed.
pyRevit CLI 4.8.5.0

eirannejad commented 3 years ago

Fixed and will be published with next release

pyrevitlabs / pyRevit

Debugging pyRevit engines and IronPython Compile Test command #1053