Our project does not build to IL2CPP

skydread1 commented 1 year ago

Problem

We are able to run the game in Unity using our generated dll with magic.

However, we are hitting an error upon building to IL2CPP:

/Applications/Unity/Hub/Editor/2021.2.3f1/Unity.app/Contents/il2cpp/build/deploy/il2cpp @Library/Bee/artifacts/rsp/8204642741344407867.rsp
Error: IL2CPP error for method 'System.Object <magic>clojure_core$load-lib__0::invokeTyped(System.Object,System.Object,clojure.lang.ISeq)' in assembly '/Users/loicblanchard/workspaces/unity-projects/hjdcdd/Library/Bee/artifacts/MacStandalonePlayerBuildProgram/ManagedStripped/clojure.core.clj.dll'
System.ArgumentException: Invalid global variables count (Parameter 'globalVariables')
   at Unity.IL2CPP.MethodBodyWriter.WriteAssignGlobalVariables(GlobalVariable[] globalVariables) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 4381
   at Unity.IL2CPP.MethodBodyWriter.WriteUnconditionalJumpTo(InstructionBlock block, Instruction target, Node currentNode) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 3219
   at Unity.IL2CPP.MethodBodyWriter.ProcessInstruction(Node node, InstructionBlock block, ResolvedInstruction ins) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 1227
   at Unity.IL2CPP.MethodBodyWriter.GenerateCodeRecursive(Node node, ReadOnlyDictionary`2 instructionBlocks) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 402
   at Unity.IL2CPP.MethodBodyWriter.GenerateCodeRecursive(Node node, ReadOnlyDictionary`2 instructionBlocks) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 448
   at Unity.IL2CPP.MethodBodyWriter.Generate() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 223
   at Unity.IL2CPP.CodeWriters.CodeWriterExtensions.WriteMethodWithMetadataInitialization(IGeneratedMethodCodeWriter writer, String methodSignature, Action`2 writeMethodBody, String uniqueIdentifier, MethodReference methodRef, Boolean writingMethodBody) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/CodeWriters/CodeWriterExtensions.cs:line 153
   at Unity.IL2CPP.MethodWriter.WriteMethodDefinition(AssemblyWriteContext context, IGeneratedMethodCodeWriter writer, MethodReference method) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodWriter.cs:line 63
   at Unity.IL2CPP.SourceWriter.WriteType(SourceWritingContext context, IGeneratedMethodCodeWriter writer, TypeReference type, NPath filePath, Boolean writeMarshalingDefinitions) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/SourceWriter.cs:line 31
   at Unity.IL2CPP.SourceWriters.SourceWriterBase`2.WriteItem(StreamWorkItemData`2 data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/SourceWriters/SourceWriterBase.cs:line 91
   at Unity.IL2CPP.Contexts.Scheduling.Streams.FileLevelParallelStreamManager`3.WorkerWriteItemsToFile(WorkItemData`2 data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/Contexts.Scheduling.Streams/FileLevelParallelStreamManager.cs:line 36
   at Unity.IL2CPP.Contexts.Scheduling.PhaseWorkScheduler`1.WorkerLoop(Object data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/Contexts.Scheduling/PhaseWorkScheduler.cs:line 287
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

All the Clojure libs were compiled using the flags mflags/*strongly-typed-invokes* and mflags/*direct-linking*.

The Project Unity version:

m_EditorVersion: 2021.2.3f1
m_EditorVersionWithRevision: 2021.2.3f1 (32358a8527b4)

nasser commented 1 year ago

Digging into the issue I am increasingly convinced that IL2CPP is unable to process unreachable CIL instructions at all, which MAGIC will occasionally emit. I've tightened up the compiler to remove two instances of unreachable code in these commits

nasser/magic@10f1a674
nasser/magic@714f5472

There are still cases where the compiler might generate unreachable instructions that are harder to change. To that end I've expanded the post-processing logic in this extension to detect and remove all unreachable instructions in every assembly in ce3b629. Hopefully that should resolve this class of issues, but I have not been able to independently verify that your code now builds as expected -- try and build again with the latest versions of MAGIC and Magic.Unity and let me know how that goes.

skydread1 commented 1 year ago

Issue

I am having 2 errors when I try to open the Unity project:

Assets/ClojureLibs/Magic.Unity.2.0.1/content/Editor/GenerateGenericWorkaroundMethods.cs(31,80): error CS0246: The type or namespace name 'clojure' could not be found (are you missing a using directive or an assembly reference?)

Assets/ClojureLibs/Magic.Unity.2.0.1/content/Editor/EliminateUnreachableInstructions.cs(76,35): error CS0246: The type or namespace name 'NotImplementedException' could not be found (are you missing a using directive or an assembly reference?)

The second error can be fixed by adding using System; at the beginning of EliminateUnreachableInstructions.cs The first error is not fixed by adding using clojure.lang at the beginning of GenerateGenericWorkaroundMethods.cs.

How to reproduce

I built the new magic version using the build pipeline flybot-sg/new-magic
I use nos command with the new dlls to recompile all my Clojure projects
I use nos to recompile Magic.Unity (I push the package to my fork which is up to date with upstream)
I packaged and pushed all the NuGet package to host repos
I nuget restore on my Unity project to download all the packages (no error here)
I started project via Unity Hub (version 2021.2.3f1)
It got errors so I opened in safe mode and got the errors shared above

nasser commented 1 year ago

Should be cleaned up, give that a shot and let me know how it goes.

I've reformatted this repository to work as a UPM package, so you can add it to the Unity package manager from the git URL or a local folder. It should also work copied into a Unity project as before.

skydread1 commented 1 year ago

We were able to load the Unity project using both the NuGet Package and the GitHub manifest dependency.

NuGet Package

In order to make the nos dotnet/nuget-push work properly, I had to move the Magic.Unity.csproj from /Runtime to the project root.

Manifest

Adding Magic.Unity from the manifest.json work right away

IL2CPP

We are still having the same globalVariables error.

AssemblyResolutionException: Failed to resolve assembly: 'Bee.BeeDriver, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name, Mono.Cecil.ReaderParameters parameters) (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Mono.Cecil.DefaultAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Mono.Cecil.MetadataResolver.Resolve (Mono.Cecil.TypeReference type) (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Mono.Cecil.ModuleDefinition.Resolve (Mono.Cecil.TypeReference type) (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Mono.Cecil.TypeReference.Resolve () (at <ee112d265aad4e098cf6cf7bf2806b0e>:0)
Magic.Unity.GenerateGenericWorkaroundMethods.collectAllReferencedAssemblies (Mono.Cecil.AssemblyDefinition assydef, System.Collections.Generic.HashSet`1[T] seen) (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/GenerateGenericWorkaroundMethods.cs:37)
Magic.Unity.GenerateGenericWorkaroundMethods.collectAllReferencedAssemblies (Mono.Cecil.AssemblyDefinition assydef, System.Collections.Generic.HashSet`1[T] seen) (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/GenerateGenericWorkaroundMethods.cs:40)
Magic.Unity.GenerateGenericWorkaroundMethods.collectAllReferencedAssemblies (Mono.Cecil.AssemblyDefinition assydef, System.Collections.Generic.HashSet`1[T] seen) (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/GenerateGenericWorkaroundMethods.cs:40)
Magic.Unity.GenerateGenericWorkaroundMethods.Init () (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/GenerateGenericWorkaroundMethods.cs:50)
Magic.Unity.IL2CPPWorkarounds.RewriteAssemblies (System.Collections.Generic.IEnumerable`1[T] files) (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/IL2CPPWorkarounds.cs:27)
Magic.Unity.IL2CPPWorkarounds.RewriteAssemblies () (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/IL2CPPWorkarounds.cs:22)
Magic.Unity.BuildPreprocessor.OnPreprocessBuild (UnityEditor.Build.Reporting.BuildReport report) (at Library/PackageCache/sr.nas.magic.unity@2f85596ae6/Editor/MagicPreprocessor.cs:28)
UnityEditor.Build.BuildPipelineInterfaces+<>c__DisplayClass16_0.<OnBuildPreProcess>b__1 (UnityEditor.Build.IPreprocessBuildWithReport bpp) (at /Users/bokken/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildPipelineInterfaces.cs:428)
UnityEditor.Build.BuildPipelineInterfaces.InvokeCallbackInterfacesPair[T1,T2] (System.Collections.Generic.List`1[T] oneInterfaces, System.Action`1[T] invocationOne, System.Collections.Generic.List`1[T] twoInterfaces, System.Action`1[T] invocationTwo, System.Boolean exitOnFailure) (at /Users/bokken/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildPipelineInterfaces.cs:402)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

/Applications/Unity/Hub/Editor/2021.2.3f1c1/Unity.app/Contents/il2cpp/build/deploy/il2cpp @Library/Bee/artifacts/rsp/4350649304388141697.rsp
Error: IL2CPP error for method 'System.Object <magic>clojure_core$load-lib__0::invokeTyped(System.Object,System.Object,clojure.lang.ISeq)' in assembly '/Users/hongheng/Projects/hjdcdd-for-read/Library/Bee/artifacts/MacStandalonePlayerBuildProgram/ManagedStripped/clojure.core.clj.dll'
System.ArgumentException: Invalid global variables count (Parameter 'globalVariables')
   at Unity.IL2CPP.MethodBodyWriter.WriteAssignGlobalVariables(GlobalVariable[] globalVariables) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 4381
   at Unity.IL2CPP.MethodBodyWriter.WriteUnconditionalJumpTo(InstructionBlock block, Instruction target, Node currentNode) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 3219
   at Unity.IL2CPP.MethodBodyWriter.ProcessInstruction(Node node, InstructionBlock block, ResolvedInstruction ins) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 1227
   at Unity.IL2CPP.MethodBodyWriter.GenerateCodeRecursive(Node node, ReadOnlyDictionary`2 instructionBlocks) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 402
   at Unity.IL2CPP.MethodBodyWriter.GenerateCodeRecursive(Node node, ReadOnlyDictionary`2 instructionBlocks) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 448
   at Unity.IL2CPP.MethodBodyWriter.Generate() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodBodyWriter.cs:line 223
   at Unity.IL2CPP.CodeWriters.CodeWriterExtensions.WriteMethodWithMetadataInitialization(IGeneratedMethodCodeWriter writer, String methodSignature, Action`2 writeMethodBody, String uniqueIdentifier, MethodReference methodRef, Boolean writingMethodBody) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/CodeWriters/CodeWriterExtensions.cs:line 153
   at Unity.IL2CPP.MethodWriter.WriteMethodDefinition(AssemblyWriteContext context, IGeneratedMethodCodeWriter writer, MethodReference method) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/MethodWriter.cs:line 63
   at Unity.IL2CPP.SourceWriter.WriteType(SourceWritingContext context, IGeneratedMethodCodeWriter writer, TypeReference type, NPath filePath, Boolean writeMarshalingDefinitions) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/SourceWriter.cs:line 31
   at Unity.IL2CPP.SourceWriters.SourceWriterBase`2.WriteItem(StreamWorkItemData`2 data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/SourceWriters/SourceWriterBase.cs:line 91
   at Unity.IL2CPP.Contexts.Scheduling.Streams.FileLevelParallelStreamManager`3.WorkerWriteItemsToFile(WorkItemData`2 data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/Contexts.Scheduling.Streams/FileLevelParallelStreamManager.cs:line 36
   at Unity.IL2CPP.Contexts.Scheduling.PhaseWorkScheduler`1.WorkerLoop(Object data) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP/Contexts.Scheduling/PhaseWorkScheduler.cs:line 287
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

BuildFailedException: Incremental Player build failed!
UnityEditor.Modules.BeeBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at /Users/bokken/buildslave/unity/build/Editor/Mono/Modules/BeeBuildPostprocessor.cs:556)
UnityEditor.OSXStandalone.OSXDesktopStandalonePostProcessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args) (at /Users/bokken/buildslave/unity/build/PlatformDependent/OSXPlayer/Extensions/Managed/OSXDesktopStandalonePostProcessor.cs:115)
UnityEditor.Modules.DefaultBuildPostprocessor.PostProcess (UnityEditor.Modules.BuildPostProcessArgs args, UnityEditor.BuildProperties& outProperties) (at /Users/bokken/buildslave/unity/build/Editor/Mono/Modules/DefaultBuildPostprocessor.cs:28)
UnityEditor.PostprocessBuildPlayer.Postprocess (UnityEditor.BuildTargetGroup targetGroup, UnityEditor.BuildTarget target, System.Int32 subtarget, System.String installPath, System.String companyName, System.String productName, System.Int32 width, System.Int32 height, UnityEditor.BuildOptions options, UnityEditor.RuntimeClassRegistry usedClassRegistry, UnityEditor.Build.Reporting.BuildReport report) (at /Users/bokken/buildslave/unity/build/Editor/Mono/BuildPipeline/PostprocessBuildPlayer.cs:370)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

Build completed with a result of 'Failed' in 54 seconds (54370 ms)
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

UnityEditor.BuildPlayerWindow+BuildMethodException: 4 errors
  at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (UnityEditor.BuildPlayerOptions options) [0x002cc] in /Users/bokken/buildslave/unity/build/Editor/Mono/BuildPlayerWindowBuildMethods.cs:193 
  at UnityEditor.BuildPlayerWindow.CallBuildMethods (System.Boolean askForBuildLocation, UnityEditor.BuildOptions defaultBuildOptions) [0x00080] in /Users/bokken/buildslave/unity/build/Editor/Mono/BuildPlayerWindowBuildMethods.cs:94 
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

The Unity version is 2021.2.3f1c1 (It is same to 2021.2.3f1)

The settings for IL2CPP are:

nasser / Magic.Unity