"Bad IL format" on self-contained linux-x64 builds if `using namespace` with same name as assembly

[ABCRic]

I'm having the same issue as #129 again on CS-Script 4.4.6.

The issue is caused by trying to CSScript.Evaluator.LoadMethod some code that has a namespace with the same name as the assembly and tries to using it, e.g. the assembly is called Thing and the code contains using Thing;. Once again fully qualifying the name bypasses the issue.

DisableReferencingFromCode = true; as suggested in #129 does not seem to help.

[oleg-shilo]

In the original #129 you have confirmed that the suggestion helped you to solve the problem:

So I left it there without trying to reproduce the problem as you had a proper solution for it. So since we are in the same position again, can you please provide a test case so I can try to reproduce the problem?

[ABCRic]

That's strange... I ended up not adding that line to the application where I'm using CS-Script and it has worked without this issue until recently.

In any case, here's the repro steps. With .NET 7 installed, in Powershell:

1. Setup

   mkdir EvalTest; cd EvalTest
   dotnet new console
   dotnet add package CS-Script --version 4.4.6

2. Replace Program.cs contents with

   
   using System;
   using CSScriptLib;

namespace EvalTest { public class Program { static void Main(string[] args) { CSScript.Evaluator.DisableReferencingFromCode = true; CSScript.RoslynEvaluator.DisableReferencingFromCode = true;

        dynamic func1 = CSScript.Evaluator.LoadMethod(@"
                public object Func()
                {{
                    return 1;
                }}");
        Console.WriteLine("Result: " + func1.Func().ToString());

        dynamic func2 = CSScript.Evaluator.LoadMethod(@"
                public object Func()
                {{
                    return EvalTest.Program.CallMe();
                }}");
        Console.WriteLine("Result: " + func2.Func().ToString());

        dynamic func3 = CSScript.Evaluator.LoadMethod(@"
                using EvalTest;
                public object Func()
                {{
                    return 3;
                }}");
        Console.WriteLine("Result: " + func3.Func().ToString());
    }

    public static int CallMe() => 2;
}

}

3. `dotnet publish --os linux --self-contained`
4. Run the binary in a linux environment. For example, using WSL:

wsl bin/Debug/net7.0/linux-x64/EvalTest

Result: 1 Result: 2 Unhandled exception. System.BadImageFormatException: Bad IL format. The format of the file '/mnt/c/temp/EvalTest/bin/debug/net7.0/linux-x64/EvalTest' is invalid. at System.Runtime.Loader.AssemblyLoadContext.g____PInvoke|5_0(IntPtr ptrNativeAssemblyBinder, UInt16 ilPath, UInt16 niPath, ObjectHandleOnStack retAssembly) at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath) at System.Reflection.Assembly.LoadFile(String path) at CSScriptLib.EvaluatorBase1.ReferenceAssembly(String assembly) at CSScriptLib.EvaluatorBase1.ReferenceAssembliesFromCode(String code, String[] searchDirs) at CSScriptLib.RoslynEvaluator.Compile(String scriptText, String scriptFile, CompileInfo info) at CSScriptLib.EvaluatorBase1.CompileCode(String scriptText, String scriptFile, CompileInfo info) at CSScriptLib.EvaluatorBase1.CompileCode(String scriptText) at CSScriptLib.EvaluatorBase1.LoadCodeByName(String scriptText, String className, Object[] args) at CSScriptLib.EvaluatorBase1.LoadMethod(String code) at EvalTest.Program.Main(String[] args) in C:\temp\EvalTest\Program.cs:line 27 Aborted

[oleg-shilo]

Thank you. I will try to look into it on weekend.

I have a question though. What is the purpose of

CSScript.Evaluator.DisableReferencingFromCode = true;
CSScript.RoslynEvaluator.DisableReferencingFromCode = true;

It doesn't actually do anything. Both lines create an instance of Roslyn evaluator, set its ReferencingFromCode to true and... never use this instance for anything at all.

Remember, CSScript.Evaluator follow the same API pattern that is used in Mono and Roslyn - the property CSScript.Evaluator always returns a new instance of the evaluator. Unless you changed this behaviour with CSScript.EvaluatorConfig.Access = EvaluatorAccess.Singleton.

Thus I am guessing it is what you were trying to achieve:

dynamic func1 = CSScript.Evaluator
                        .With(evaluator => evaluator.DisableReferencingFromCode = true)
                        .LoadMethod(@"
                                     public object Func()
                                     {{
                                         return 1;
                                     }}");

Though... you don't need to disable referencing from code as your code does not have any referencing directives. I am also not sure about those double brackets {{.

As for the actual error, I think the problem might be caused by the self-contained assembly to load itself in its own AppDomain. I do not trust the error message and think if you check the actual assembly /mnt/c/temp/EvalTest/bin/debug/net7.0/linux-x64/EvalTest you will find that it is of the right CPU architecture. you just cannot load it as it is linked as self-contained.

It's interesting to see if .With(evaluator => evaluator.ReferenceDomainAssemblies = false) may help.

Though it is my guess only and it needs to be checked

[oleg-shilo]

Interestingly enough I have managed to have your test ruining right away:

// See https://aka.ms/new-console-template for more information
using System;
using CSScriptLib;

dynamic func1 = CSScript.Evaluator.LoadMethod(@"
                    public object Func()
                    {
                        return 1;
                    }");
Console.WriteLine("Result: " + func1.Func().ToString());

dynamic func2 = CSScript.Evaluator

.LoadMethod(@"
                    public object Func()
                    {
                        return Foo.CallMe();
                    }");
            Console.WriteLine("Result: " + func2.Func().ToString());

public class Foo
{
    public static int CallMe() => 2;
} 

I have attached the project EvalTest.zip

[ABCRic]

Remember, CSScript.Evaluator follow the same API pattern that is used in Mono and Roslyn - the property CSScript.Evaluator always returns a new instance of the evaluator. Unless you changed this behaviour with CSScript.EvaluatorConfig.Access = EvaluatorAccess.Singleton.

Thanks for pointing this out, I was not aware of this. The With(evaluator => evaluator.DisableReferencingFromCode = true) solution does prevent the error from happening.

It's interesting to see if .With(evaluator => evaluator.ReferenceDomainAssemblies = false) may help.

ReferenceDomainAssemblies is a method, I assume you meant With(evaluator => evaluator.ReferenceDomainAssemblies(DomainAssemblies.None)), which from what I can see does not make a difference.

Interestingly enough I have managed to have your test ruining right away:

(snip)

Your example does not have the failing case func3, though from the way I posted it it is not clear why that case exists in the first place, my apologies. Allow me to elaborate:

• func1 - just tests if it works:

                  public object Func()
                  {
                      return 1;
                  }

• func2 - tests a fully qualified reference

  
                  public object Func()
                  {
                      return EvalTest.Program.CallMe();
                  }

• func3 - tests whether using EvalTest causes any issues (it does)

                  using EvalTest;
                  public object Func()
                  {
                      return 3;
                  }

• not shown - the actual use case for doing this

                  using EvalTest;
                  public object Func()
                  {
                      return Program.CallMe();
                  }

The reason I want to do this is the actual code (passed into LoadMethod) in my production application is created at runtime and I want to, as a convenience, be able to reference code from the application's main namespace without having to full qualify it. (your test has the Foo class unnamespaced so it does not have this issue, but this is not an option in my application)

Assuming that the evaluator's context has all the assemblies that the application depends on (which seems to be the case), then for my use case DisableReferencingFromCode = false works since I don't need the functionality it is disabling - i.e. I do not need to load new assemblies at runtime.

That being said, I do believe it is as you said here:

you will find that it is of the right CPU architecture. you just cannot load it as it is linked as self-contained.

In a self-contained build (in this example) ./EvalTest is a linux executable binary and the actual assembly is at EvalTest.dll. As far as I am aware, for an assembly named XYZ:

• on Windows, the assembly file will be named XYZ.dll and if the assembly is executable there will be an additional, executable binary file named XYZ.exe
• on Linux, the assembly file will be named XYZ.dll and if the assembly is executable there will be an additional, executable binary file named XYZ

This file layout is the same regardless of if self-contained publish was used or not. The difference is the executable file is either a self-sufficient .NET distribution in the first case, and a stub that calls into the system-installed .NET runtime in the second.

Actually, now that I'm thinking through this, the issue should occur for the linux publish even without the --self-contained option. I have just tested this and it does happen, with step 3 being replaced with just dotnet publish --os linux.

Given this, I am a bit confused as to why CS-Script attempts to load the XYZ file as an assembly; shouldn't it be trying XYZ.dll instead?

Testing even further, a publish targeting windows (dotnet publish --os win) is initially fine but creating a file with the same name but no extension (EvalTest in this case), with some dummy text inside, causes the following exception:

Unhandled exception. CSScriptLib.CompilerException: error CS0009: Metadata file 'C:\temp\EvalTest\bin\Debug\net7.0\win-x64\EvalTest' could not be opened -- PE image doesn't contain managed metadata.

   at CSScriptLib.RoslynEvaluator.Compile(String scriptText, String scriptFile, CompileInfo info)
   at CSScriptLib.EvaluatorBase`1.CompileCode(String scriptText, String scriptFile, CompileInfo info)
   at CSScriptLib.EvaluatorBase`1.CompileCode(String scriptText)
   at CSScriptLib.EvaluatorBase`1.LoadCodeByName(String scriptText, String className, Object[] args)
   at CSScriptLib.EvaluatorBase`1.LoadMethod(String code)
   at EvalTest.Program.Main(String[] args) in C:\temp\EvalTest\Program.cs:line 25

 

So to sum it all up, I think the issue is as follows:

1. using XYZ causes an attempt to load assembly XYZ
2. this process tries to load a file XYZ (which on linux fails, but on windows that file does not exist)
3. next it tries to load XYZ.dll (or maybe XYZ.exe, I'm not sure).

My question is: why is step 2 attempted at all? The only reason I see for it is so that it loads up a binary on linux, but it seems to be trying to load the binary as a .NET assembly (hence the Bad IL format message), which fails. Is there some other case where it makes sense to load a file with the name of the assembly but with no extension?

[oleg-shilo]

Thank you for such a comprehensive explanation. Most likely loading XYZ instead of XYZ.dll is a side effect of the default assembly probing algorithm. Both files are valid candidates for probing but the DLL should be probed first (though it's not) and even the failure of the probe should not abort probing. Thus seems like a problem.

Will have a look. Will also check the Func3 case.

[oleg-shilo]

• Re: func3 Please check the attached sample. Seems to work. EvalTest2.zip

• XYZ vs XYZ.dll It is an extremely interesting problem. When dotnet publishes the project it creates the executable that is "invalid assembly". Invalid because it does not have metadata. But the problem is that cs-script checks if the assembly can be loaded (with Assembly.Load) before it uses it for referencing. And... it does not fail, even though it fails later when compiler tries to use it.

[oleg-shilo]

I have fixed it by preferring files with the extension if the reference from code (e.g. using XYZ;) has no extension but the actual file with the extension in fact exists. (875c87d)

The fix will be available in the very next release.