Cannot use ExpandoObject as 'globals' parameter for Script.Run()

[Stmated]

How to reproduce:

var script = CSharpScript.Create("a + b");

dynamic expando = new ExpandoObject();
var dictionary = (IDictionary<String, Object>) expando;
dictionary.Add("a", 3);
dictionary.Add("b", 4);
script.Run(expando);

Gives error message: The name 'b' does not exist in the current context

Is there another, recommended way of giving a dynamically built object as the globals parameter to the script?

[Stmated]

The cause of this not being supported can be found in https://github.com/dotnet/roslyn/blob/03e30451ce7eb518e364b5806c524623424103e4/src/Scripting/Core/ScriptVariables.cs#L98 where it finds the fields and properties of the given instance and converts it into a map of script variables.

If this could check if the object is an IDictionary<String, Object> object, then my scenario could be supported quite easily (in my opinion, at least).

Or am I completely off-base? I don't know the codebase well-enough to actually be confident.

[tmat]

@Stmated Thanks for letting us know that this feature is important for you. Currently the ability to use a dynamic object as a host object is not supported. We have it on backlog.

BTW, the spot you identified is not the only place that would need to implement support for dynamic variables.

[codespare]

Yes, launching csi.exe /r:System.Dynamic (as of revision 232206a6fdabd437ccd64d185e3938db47030b16) and typing:

using System.Dynamic;
var i = 1;
dynamic d = i;
d.ToString();

fails with error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create' for instance. Every access to a dynamic variable members seem to fail with that error from what I can tell. Great to see the REPL in csi.exe though :tada:

[tmat]

The command line processing is not all working yet. Try:

> #r "System.Dynamic"
> #r "Microsoft.CSharp"
> dynamic d = 1;
> d.ToString()
"1"

[tmat]

(btw, these assemblies should be available by default).

[codespare]

Great, thanks! Not the first time I forgot Microsoft.CSharp.

[Tiliavir]

@Stmated did you find a workaround for that? Or @tmat is there any status update on the backlog item?

We would really need something like this, to be able to provide globals dynamically... We also tried to create a type dynamically as suggested here: http://stackoverflow.com/questions/29413942/c-sharp-anonymous-object-with-properties-from-dictionary/29428640#29428640, but this was not working (probably caused by #2246).

Thanks for your feedback, Markus

[tmat]

No update, it's still on backlog.

An alternative workaround would be to add an indirection:

var script = CSharpScript.Create("dyn.a + dyn.b");
...
script.Run(new { dyn = expando })

[Tiliavir]

@tmat: Thank you very much, that is a nice workaround! I will rewrite my code until this is implemented!

[discosultan]

@tmat Thanks for the workaround! This topic has provided a lot of help regarding importing dynamic global to the Roslyn scripting context.

The entire scripting API has been a pleasure to work with except for this little missing feature. I assume there is not update on this backlog item. Is there a way to vote for this feature?

In my scenario, I am providing an in-game REPL console where users can dynamically add variables to be modified at runtime. While the workaround works, it is causing a lot of headache with the current console's autocompletion system and generally doesn't "feel" nice.

[wiikka]

@tmat what is status of this one? Would really need the possibility for dynamic global without above workaround!

[davidpurkiss]

Also looking for a status update? I've implemented the workaround but would prefer not having to access 'globals' from another member.

[CrosserTechnologies]

Any update on this issue, or is it still on the backlog? edit: solved our needs with the example below

// for sending in parameters to the script
public class Globals
{
    public dynamic data;
}

// Script that will use dynamic
var scriptContent = "data.X + data.Y";

// data to be sent into the script
dynamic expando = new ExpandoObject();
expando.X = 34;
expando.Y = 45;

// setup references needed
var refs = new List<MetadataReference>{
    MetadataReference.CreateFromFile(typeof(Microsoft.CSharp.RuntimeBinder.RuntimeBinderException).GetTypeInfo().Assembly.Location),
    MetadataReference.CreateFromFile(typeof(System.Runtime.CompilerServices.DynamicAttribute).GetTypeInfo().Assembly.Location)};
var script = CSharpScript.Create(scriptContent, options: ScriptOptions.Default.AddReferences(refs), globalsType: typeof(Globals));

script.Compile();

// create new global that will contain the data we want to send into the script
var g = new Globals() { data = expando };

//Execute and display result
var r = script.RunAsync(g).Result;
Console.WriteLine(r.ReturnValue);

[schprl]

When running the above example I get

Microsoft.CodeAnalysis.Scripting.CompilationErrorException: "(1,1): error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.Binder.BinaryOperation'"

which is in Microsoft.CSharp.dll Did this happen to anyone else?

[davidpurkiss]

@schprl We've just had an issue regarding dynamic objects and Microsoft.CSharp.dll. We had to remove the Microsoft.CSharp.dll assembly reference to get dynamics working. Might be related.

[dtoppani-twist]

Just tried to use ExpandoObject today and it appears this is still not supported. However as I still need to accept on-demand globals just before the script is executed, I am currently trying the following workaround:

• create a script string that declares the variables needed with default values. The assembly do need to be imported which can be found through reflection. For example, if the variable was an integer named test, the generated script would be int test = default(int);
• run this generated script
• use the returned state and update the variables to the desired values
• continue with the actual script

[DeluxeAgency2020]

Why Roslyn is not supporting ExpandoObject or Anonymous. Is there any reason?

[luisfb]

Issue opened on 30 May 2015. We are on Dec 2021 and still no fix. What a shame.

[TheSnowfield]

Still waiting for the fix in the 2022.

[TheSnowfield]

After some work, I have got another solution for avoiding using the ExpandoObject. It can run the script in a persistent global context, works perfectly like csi.exe.

I have created a gist with some notes, please reference: https://gist.github.com/TheSnowfield/2c52641d58e73ade1df2447c15f48683

Hope it helps! :3

[stonstad]

Using the example above @CrosserTechnologies I find that it fails with this error in .NET 6.0:

Microsoft.CodeAnalysis.Scripting.CompilationErrorException: '(1,1): error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create''

The Microsoft.CSharp library is included.

[stonstad]

Not sure who to ask. @jcouv @jaredpar Could we please get a refusal or acceptance for this capability on the roadmap? Related: https://github.com/dotnet/roslyn/issues/3194#issuecomment-107103175

[tmat]

@stonstad We think this feature would be useful but we have been prioritizing other areas over scripting, so it stays on backlog at this point.

[stonstad]

Thanks @tmat. Are there any known work-arounds to supporting dynamic that are functional in .NET 6.0?

[tmat]

Depends what exactly you want to achieve. One thing that might work for you is to add an indirection like so:

class MyGlobals
{
   public readonly ExpandoObject DynamicGlobals = ...;
}

and then in your script write:

DynamicGlobals.a + DynamicGlobals.b

[stonstad]

Depends what exactly you want to achieve. One thing that might work for you is to add an indirection like so:

class MyGlobals
{
   public readonly ExpandoObject DynamicGlobals = ...;
}

and then in your script write:

DynamicGlobals.a + DynamicGlobals.b

I'm not aware of any approach that works in .NET 6.0. This code, which I believe is in line with your approach, fails:

public static async Task Test()
{
   // script
   ScriptOptions scriptOptions = ScriptOptions.Default;
   var refs = new List<MetadataReference>{
      MetadataReference.CreateFromFile(typeof(RuntimeBinderException).GetType().Assembly.Location),
      MetadataReference.CreateFromFile(typeof(DynamicAttribute).GetType().Assembly.Location),
       MetadataReference.CreateFromFile(typeof(DynamicObject).GetType().Assembly.Location),
       MetadataReference.CreateFromFile(typeof(CSharpArgumentInfo).GetType().Assembly.Location),
       MetadataReference.CreateFromFile(typeof(ExpandoObject).GetType().Assembly.Location)
    };

   scriptOptions.AddReferences(refs);
   string scriptContents = "DynamicGlobals.a + DynamicGlobals.b";
   Script script = CSharpScript.Create(scriptContents, scriptOptions, typeof(MyGlobals));
   script.Compile();

   MyGlobals globals = new MyGlobals();

   IDictionary<string, object> globalsDictionary = globals.DynamicGlobals as IDictionary<string, object>;
   globalsDictionary.Add("a", 1);
   globalsDictionary.Add("b", 1);

   ScriptState scriptState = await script.RunAsync(globals); // error (below)
   Console.WriteLine(scriptState.ReturnValue);
}

Microsoft.CodeAnalysis.Scripting.CompilationErrorException: '(1,16): error CS1061: 'ExpandoObject' does not contain a definition for 'a' and no accessible extension method 'a' accepting a first argument of type 'ExpandoObject' could be found (are you missing a using directive or an assembly reference?)'

Is there a small detail I am missing that allows it to work?

[stonstad]

Hey @tmat Just reaching out to ask if there is a known workflow that supports .NET 6.0, see above failure which indicates this may not be possible.

[yueyinqiu]

Anything new about this?

Actually I can't even do this (I think this shares the same cause):

using Microsoft.CodeAnalysis.CSharp.Scripting;

var globals = new Globals();
await CSharpScript.EvaluateAsync("X+1", globals: globals);

public class Globals
{
    public dynamic X { get; set; } = 1;
}

It throws Microsoft.CodeAnalysis.Scripting.CompilationErrorException saying error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create'.

[MattMinke]

I found a solution to be able to use ExpandoObject and dynamic. It needs some cleanup before it will be production ready, but shows how to accomplish what is being asked.

using Microsoft.CodeAnalysis.CSharp.Scripting;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Scripting.Hosting;
using Microsoft.CodeAnalysis;
using System.Collections.Immutable;
using System.Reflection;
using Microsoft.Extensions.DependencyModel;
using System.Dynamic;

namespace MySample
{
    internal class Program
    {
        static async Task Main(string[] args)
        {
            try
            {
                await new Example().Execute();
            }
            catch (Exception ex)
            {
                throw;
            }
        }
    }

    /// <summary>
    /// Using ReflectionEmit with Microsoft.CodeAnalysis.CSharp.Scripting is not supported. 
    /// The workaround is to use CSharpCompilation instead.
    /// ref: https://github.com/dotnet/roslyn/issues/2246
    /// ref: https://github.com/dotnet/roslyn/pull/6254
    /// </summary>
    public class CSharpCompilationScriptGlobalTypeBuilder
    {

        private const string TEMPLATE = @"
using System;
using System.Collections.Generic;
public class {0}
{{
    public {0}(
        IDictionary<string, Object> extensions)
    {{
        {1}
    }}
    {2}
}}";

        private int unique = 0;
        private readonly IDictionary<Guid, GlobalTypeInfo> _cache;

        public CSharpCompilationScriptGlobalTypeBuilder()
        {
            _cache = new Dictionary<Guid, GlobalTypeInfo>();
        }

        private static PortableExecutableReference GetMetadataReference(Type type)
        {
            var assemblyLocation = type.Assembly.Location;
            return MetadataReference.CreateFromFile(assemblyLocation);
        }

        public GlobalTypeInfo Create(Guid key, IDictionary<string, object> extensions)
        {
            // No locking. the worst that happens is we generate the type
            // multiple times and throw all but one away. 
            if (!_cache.TryGetValue(key, out var item))
            {
                item = CreateCore(key, extensions.ToDictionary(x => x.Key, x => x.Value.GetType()));
                _cache[key] = item;
            }
            return item;
        }

        private GlobalTypeInfo CreateCore(Guid key, IDictionary<string, Type> extensionDetails)
        {
            var count = Interlocked.Increment(ref unique);
            var typeName = $"DynamicType{count}";

            var code = String.Format(TEMPLATE,
                typeName,
                String.Join(System.Environment.NewLine, extensionDetails.Select(pair => $"{pair.Key} = ({pair.Value.FullName})extensions[\"{pair.Key}\"];")),
                String.Join(System.Environment.NewLine, extensionDetails.Select(pair => $"public {pair.Value.FullName} {pair.Key} {{ get; }}")));

            SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(code);

            //TODO: need to change the way references are being added.
            // See these links for how the preferred method:
            // ref: https://github.com/dotnet/roslyn/issues/49498
            // ref: https://github.com/dotnet/roslyn/issues/49498#issuecomment-776059232
            // ref: https://github.com/jaredpar/basic-reference-assemblies
            // ref: https://stackoverflow.com/q/32961592/2076531
            PortableExecutableReference[] references = Assembly.GetEntryAssembly().GetReferencedAssemblies()
                .Select(a => MetadataReference.CreateFromFile(Assembly.Load(a).Location))
                .Concat(extensionDetails.Values.Select(GetMetadataReference))
                .Append(GetRuntimeSpecificReference())
                .Append(GetMetadataReference(typeof(CSharpCompilationScriptGlobalTypeBuilder)))
                .Append(GetMetadataReference(typeof(System.Linq.Enumerable)))
                .Append(GetMetadataReference(typeof(System.Runtime.AssemblyTargetedPatchBandAttribute)))
                .Append(GetMetadataReference(typeof(Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo)))
                .Append(GetMetadataReference(typeof(System.Collections.Generic.IDictionary<string, object>)))
                .Append(GetMetadataReference(typeof(object)))
                .Append(GetMetadataReference(typeof(GlobalTypeInfo)))
                .ToArray();

            Compilation compilation = CSharpCompilation.Create(
                $"ScriptGlobalTypeBuilder{count}", new[] { syntaxTree }, references,
                new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

            ImmutableArray<byte> assemblyBytes = compilation.EmitToArray();
            PortableExecutableReference libRef = MetadataReference.CreateFromImage(assemblyBytes);
            Assembly assembly = Assembly.Load(assemblyBytes.ToArray());

            return new GlobalTypeInfo()
            {
                Key = key,
                Assembly = assembly,
                Reference = libRef,
                Type = assembly.GetType(typeName),
            };
        }

        private static PortableExecutableReference GetRuntimeSpecificReference()
        {
            var assemblyLocation = typeof(object).Assembly.Location;
            var runtimeDirectory = Path.GetDirectoryName(assemblyLocation);
            var libraryPath = Path.Join(runtimeDirectory, @"netstandard.dll");

            return MetadataReference.CreateFromFile(libraryPath);
        }
    }

    public class Example
    {
        public async Task Execute()
        {
            var _factory = new CSharpCompilationScriptGlobalTypeBuilder();
            string script = @"(Global1.Number+WhatEverNameIWant.Number).ToString() + Global1.Text + WhatEverNameIWant.Text";
            //TODO: need to generate an Id for this script. dealers choice
            Guid key = Guid.NewGuid();

            //TODO: populate this dictionary with the globals you want. 
            //Dictionary<string, object> globals = new Dictionary<string, object>
            //{
            //    { "Global1", new MyCoolClass() { Number = 100, Text = "Something" } },
            //    { "WhatEverNameIWant", new MyCoolClass() { Number = 500, Text = "Longer Text Value" } }
            //};

            dynamic globals = new ExpandoObject();
            globals.Global1 = new MyCoolClass() { Number = 100, Text = "Something" };
            globals.WhatEverNameIWant = new MyCoolClass() { Number = 500, Text = "Longer Text Value" };

            // Act: 
            GlobalTypeInfo typeInfo = _factory.Create(key, globals);

            // TODO: Ideally you would cache the runner for reuse instead of creating it each time.
            var runner = await CreateRunnerAsync(script, typeInfo);
            var instance = Activator.CreateInstance(typeInfo.Type, new object[] { globals });
            var result = await runner.Invoke(instance);
            Console.Write(result);
        }

        private async Task<Microsoft.CodeAnalysis.Scripting.ScriptRunner<string>> CreateRunnerAsync(
            string scriptContent,
            GlobalTypeInfo typeInfo)
        {

            //ref: https://github.com/dotnet/roslyn/blob/main/docs/wiki/Scripting-API-Samples.md
            var options = Microsoft.CodeAnalysis.Scripting.ScriptOptions.Default
                .AddImports("System")
                .AddImports("System.Text");

            //TODO: this is overkill find a better way to do this.
            var assemblies = Assemblies.ApplicationDependencies()
                .SelectMany(assembly => assembly.GetExportedTypes())
                .Select(type => type.Assembly)
                .Distinct();
            options.AddReferences(assemblies);

            using (var loader = new InteractiveAssemblyLoader())
            {
                loader.RegisterDependency(typeInfo.Assembly);

                var script = CSharpScript.Create<string>(
                    scriptContent,
                    options.WithReferences(typeInfo.Reference),
                    globalsType: typeInfo.Type,
                    assemblyLoader: loader);

                return script.CreateDelegate();
            }
        }
    }

    public class GlobalTypeInfo
    {
        public Assembly Assembly { get; init; }
        public MetadataReference Reference { get; init; }
        public Type Type { get; init; }
        public Guid Key { get; init; }
    }
    public class MyCoolClass
    {
        public string Text { get; set; }
        public int Number { get; set; }
    }
}

namespace System.Reflection
{
    public static class Assemblies
    {
        public static IEnumerable<Assembly> ApplicationDependencies(Func<AssemblyName, bool> predicate = null)
        {
            if (predicate == null)
            {
                predicate = _ => true;
            }
            try
            {
                return FromDependencyContext(DependencyContext.Default, predicate);
            }
            catch
            {
                // Something went wrong when loading the DependencyContext, fall
                // back to loading all referenced assemblies of the entry assembly...
                return FromAssemblyDependencies(Assembly.GetEntryAssembly(), predicate);
            }
        }

        private static IEnumerable<Assembly> FromDependencyContext(
            DependencyContext context, Func<AssemblyName, bool> predicate)
        {
            var assemblyNames = context.RuntimeLibraries
                .SelectMany(library => library.GetDefaultAssemblyNames(context));

            return LoadAssemblies(assemblyNames, predicate);
        }

        private static IEnumerable<Assembly> FromAssemblyDependencies(Assembly assembly, Func<AssemblyName, bool> predicate)
        {
            var dependencyNames = assembly.GetReferencedAssemblies();

            var results = LoadAssemblies(dependencyNames, predicate);

            if (predicate(assembly.GetName()))
            {
                results.Prepend(assembly);
            }

            return results;
        }

        private static IEnumerable<Assembly> LoadAssemblies(IEnumerable<AssemblyName> assemblyNames, Func<AssemblyName, bool> predicate)
        {
            var assemblies = new List<Assembly>();

            foreach (var assemblyName in assemblyNames.Where(predicate))
            {
                try
                {
                    // Try to load the referenced assembly...
                    assemblies.Add(Assembly.Load(assemblyName));
                }
                catch
                {
                    // Failed to load assembly. Skip it.
                }
            }

            return assemblies;
        }
    }
}

namespace Microsoft.CodeAnalysis
{
    public static class CompilationExtensions
    {
        public static ImmutableArray<byte> EmitToArray(this Compilation compilation)
        {
            using (MemoryStream assemblyStream = new MemoryStream())
            {

                Microsoft.CodeAnalysis.Emit.EmitResult emitResult = compilation.Emit(assemblyStream);

                if (emitResult.Success)
                {
                    return ImmutableArray.Create<byte>(assemblyStream.ToArray());
                }

                var errors = emitResult
                    .Diagnostics
                    .Select(diagnostic => diagnostic.GetMessage())
                    .Select(message => new Exception(message));

                throw new AggregateException(errors);
            }
        }
    }
}