microsoft / ClearScript

A library for adding scripting to .NET applications. Supports V8 (Windows, Linux, macOS) and JScript/VBScript (Windows).
https://microsoft.github.io/ClearScript/
MIT License
1.74k stars 148 forks source link

how to use node_modules in ClearScript #469

Closed stone89son closed 1 year ago

stone89son commented 1 year ago

Hi everyone. I'm wanting to use the node_modules for my application. I my host code:

  _engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDynamicModuleImports)
            {

                DefaultAccess = ScriptAccess.Full,
                SuppressExtensionMethodEnumeration = true,
                AllowReflection = true,
                DocumentSettings=new DocumentSettings()
                {

                    SearchPath = Constant.NodeModulesPath,
                    AccessFlags = DocumentAccessFlags.EnableFileLoading
                },
            };

Client code:

import * as _ from 'lodash';
_.defaults({ 'a': 1 }, { 'a': 3, 'b': 2 });

When I executed and bellow details for exception:,

{
  "ClassName": "Microsoft.ClearScript.ScriptEngineException",
  "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
  "Data": null,
  "InnerException": {
    "ClassName": "Microsoft.ClearScript.ScriptEngineException",
    "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
    "Data": null,
    "InnerException": {
      "ClassName": "System.IO.FileLoadException",
      "Message": null,
      "Data": null,
      "InnerException": {
        "ClassName": "System.IO.FileNotFoundException",
        "Message": "Could not load file 
ClearScriptLib commented 1 year ago

Hi @stone89son,

ClearScript doesn't implement Node.js functionality such as node_modules path resolution or package.json support. However, if you've installed an npm package, you can use its modules from ClearScript with some setup.

Here's an example. Suppose you've installed the Lodash standard modules via npm install -g lodash-es. You can then do this:

var appDataPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData);
var nodeModulesPath = Path.Combine(appDataPath, "npm", "node_modules");
_engine.DocumentSettings.AccessFlags = DocumentAccessFlags.EnableFileLoading;
_engine.DocumentSettings.SearchPath = Path.Combine(nodeModulesPath, "lodash-es");

And now you can import the Lodash modules:

_engine.AddHostType(typeof(Console));
_engine.Execute(new DocumentInfo { Category = ModuleCategory.Standard }, @"
    import * as _ from 'lodash';
    const result = _.defaults({ 'a': 1 }, { 'a': 3, 'b': 2 });
    Console.WriteLine(JSON.stringify(result));
");

This code produces the output {"a":1,"b":2}, as expected. Keep in mind that, unfortunately, many npm packages depend on Node.js or Web APIs and therefore can't be used from ClearScript without modification or additional setup.

Good luck!

ClearScriptLib commented 1 year ago

Please reopen this issue if you have additional questions or comments about this topic. Thank you!

stone89son commented 1 year ago

thank you for reply and help me! sorry, i'm tried, no working. in the case use for multiple module in node-module. example: lodash, underscore... fixed for location: _engine.DocumentSettings.SearchPath = Path.Combine(nodeModulesPath, "lodash-es"); it is work?

ClearScriptLib commented 1 year ago

Hi @stone89son,

You can do something like this:

_engine.DocumentSettings.SearchPath = Path.Combine(nodeModulesPath, "lodash-es");
_engine.DocumentSettings.SearchPath += ";" + Path.Combine(nodeModulesPath, "underscore");

And then:

import * as lodash from 'lodash';
import * as underscore from 'underscore-esm';
Console.WriteLine(JSON.stringify(lodash.defaults({ 'a': 1 }, { 'a': 3, 'b': 2 })));
Console.WriteLine(underscore.keys({ foo: 123, bar: 456, baz: 789 }).toString());

Good luck!

stone89son commented 1 year ago

In Host Code:

            _engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDynamicModuleImports)
            {

                DefaultAccess = ScriptAccess.Full,
                SuppressExtensionMethodEnumeration = true,
                AllowReflection = true,
                DocumentSettings=new DocumentSettings()
                {

                    SearchPath =  "D:\\Projects\\others\\GCast\\GCastV8\\bin\\Debug\\node_modules\\lodash\\lodash.js".Replace("\\", "/"),
                    AccessFlags = DocumentAccessFlags.EnableFileLoading
                },
            };

            _engine.EnableRuntimeInterruptPropagation = true;
            _engine.SuppressExtensionMethodEnumeration = true;

In Client Code: import * as lodash from 'lodash';

And output error:

{
  "ClassName": "Microsoft.ClearScript.ScriptEngineException",
  "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
  "Data": null,
  "InnerException": {
    "ClassName": "Microsoft.ClearScript.ScriptEngineException",
    "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
    "Data": null,
    "InnerException": {
      "ClassName": "System.IO.FileLoadException",
      "Message": null,
      "Data": null,
      "InnerException": {
        "ClassName": "System.IO.FileNotFoundException",
        "Message": "Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
        "Data": null,
        "InnerException": null,
        "HelpURL": null,
        "StackTraceString": "   at Microsoft.ClearScript.DefaultDocumentLoader.<LoadDocumentAsync>d__22.MoveNext()",
        "RemoteStackTraceString": null,
        "RemoteStackIndex": 0,
        "ExceptionMethod": "8\nMoveNext\nClearScript.Core, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.DefaultDocumentLoader+<LoadDocumentAsync>d__22\nVoid MoveNext()",
        "HResult": -2147024894,
        "Source": "ClearScript.Core",
        "WatsonBuckets": null,
        "FileNotFound_FileName": "lodash",
        "FileNotFound_FusionLog": null
      },
      "HelpURL": null,
      "StackTraceString": "   at Microsoft.ClearScript.DocumentLoader.LoadDocument(DocumentSettings settings, Nullable`1 sourceInfo, String specifier, DocumentCategory category, DocumentContextCallback contextCallback)\r\n   at Microsoft.ClearScript.DocumentSettings.LoadDocument(Nullable`1 sourceInfo, String specifier, DocumentCategory category, DocumentContextCallback contextCallback)\r\n   at Microsoft.ClearScript.V8.V8ProxyHelpers.LoadModule(IntPtr pSourceDocumentInfo, String specifier, DocumentCategory category, UniqueDocumentInfo& documentInfo)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyManaged.LoadModule(IntPtr pSourceDocumentInfo, Ptr pSpecifier, Ptr pResourceName, Ptr pSourceMapUrl, UInt64& uniqueId, Boolean& isModule, Ptr pCode, IntPtr& pDocumentInfo)",
      "RemoteStackTraceString": null,
      "RemoteStackIndex": 0,
      "ExceptionMethod": "8\nLoadDocument\nClearScript.Core, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.DocumentLoader\nMicrosoft.ClearScript.Document LoadDocument(Microsoft.ClearScript.DocumentSettings, System.Nullable`1[Microsoft.ClearScript.DocumentInfo], System.String, Microsoft.ClearScript.DocumentCategory, Microsoft.ClearScript.DocumentContextCallback)",
      "HResult": -2146232799,
      "Source": "ClearScript.Core",
      "WatsonBuckets": null,
      "FileLoad_FileName": "lodash",
      "FileLoad_FusionLog": null
    },
    "HelpURL": null,
    "StackTraceString": "   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyNative.Invoke[T](Func`2 func)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8ContextProxyImpl.Execute(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ExecuteRaw(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ExecuteInternal(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass119_0.<Execute>b__0()\r\n   at Microsoft.ClearScript.ScriptEngine.ScriptInvokeInternal[T](Func`1 func)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass126_0`1.<ScriptInvoke>b__0()\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyManaged.InvokeHostAction(IntPtr pAction)",
    "RemoteStackTraceString": null,
    "RemoteStackIndex": 0,
    "ExceptionMethod": "8\nInvoke\nClearScript.V8, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.V8.SplitProxy.V8SplitProxyNative\nT Invoke[T](System.Func`2[Microsoft.ClearScript.V8.SplitProxy.IV8SplitProxyNative,T])",
    "HResult": -2146233079,
    "Source": "ClearScript.V8",
    "WatsonBuckets": null,
    "ScriptEngineName": "V8ScriptEngine",
    "ScriptErrorDetails": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.\n    at <anonymous>:0:0",
    "IsFatal": false,
    "ExecutionStarted": true
  },
  "HelpURL": null,
  "StackTraceString": "   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyNative.Invoke(Action`1 action)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8ContextProxyImpl.InvokeWithLock(Action action)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ScriptInvoke[T](Func`1 func)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.Execute(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.ScriptEngine.Execute(DocumentInfo documentInfo, String code)\r\n   at GCastV8.Engine.GCastEngine.Execute(String pathScript) in D:\\Projects\\others\\GCast\\GCastV8\\Engine\\GCastEngine.cs:line 253",
  "RemoteStackTraceString": null,
  "RemoteStackIndex": 0,
  "ExceptionMethod": "8\nInvoke\nClearScript.V8, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.V8.SplitProxy.V8SplitProxyNative\nVoid Invoke(System.Action`1[Microsoft.ClearScript.V8.SplitProxy.IV8SplitProxyNative])",
  "HResult": -2146233079,
  "Source": "ClearScript.V8",
  "WatsonBuckets": null,
  "ScriptEngineName": "V8ScriptEngine",
  "ScriptErrorDetails": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.\n    at <anonymous>:0:0",
  "IsFatal": false,
  "ExecutionStarted": true
}
ClearScriptLib commented 1 year ago

Hi @stone89son,

Please see the sample code above. SearchPath should only specify directory paths. Including a file name such as "lodash.js" makes the SearchPath entry unusable.

Thanks!

stone89son commented 1 year ago

Please see the sample code above. SearchPath should only specify directory paths. Including a file name such as "lodash.js" makes the SearchPath entry unusable.

I tried multicase. Enviroment:

  1. Microsoft.ClearScript 7.3.6.
  2. Visual studio 2022 Pro
  3. Net Framework 4.8
  4. Win 10 Pro

In the your case: Host Code:

     _engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDynamicModuleImports)
            {

                DefaultAccess = ScriptAccess.Full,
                SuppressExtensionMethodEnumeration = true,
                AllowReflection = true,
                DocumentSettings=new DocumentSettings()
                {

                    SearchPath =Path.Combine("D:\\Projects\\others\\GCast\\GCastV8\\bin\\Debug\\node_modules", "lodash-es"),
                    AccessFlags = DocumentAccessFlags.EnableFileLoading
                },
            };

Client Code: import * as lodash from 'lodash'; Node module: dir

Error details:

14:51:19    {
  "ClassName": "Microsoft.ClearScript.ScriptEngineException",
  "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
  "Data": null,
  "InnerException": {
    "ClassName": "Microsoft.ClearScript.ScriptEngineException",
    "Message": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
    "Data": null,
    "InnerException": {
      "ClassName": "System.IO.FileLoadException",
      "Message": null,
      "Data": null,
      "InnerException": {
        "ClassName": "System.IO.FileNotFoundException",
        "Message": "Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.",
        "Data": null,
        "InnerException": null,
        "HelpURL": null,
        "StackTraceString": "   at Microsoft.ClearScript.DefaultDocumentLoader.<LoadDocumentAsync>d__22.MoveNext()",
        "RemoteStackTraceString": null,
        "RemoteStackIndex": 0,
        "ExceptionMethod": "8\nMoveNext\nClearScript.Core, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.DefaultDocumentLoader+<LoadDocumentAsync>d__22\nVoid MoveNext()",
        "HResult": -2147024894,
        "Source": "ClearScript.Core",
        "WatsonBuckets": null,
        "FileNotFound_FileName": "lodash",
        "FileNotFound_FusionLog": null
      },
      "HelpURL": null,
      "StackTraceString": "   at Microsoft.ClearScript.DocumentLoader.LoadDocument(DocumentSettings settings, Nullable`1 sourceInfo, String specifier, DocumentCategory category, DocumentContextCallback contextCallback)\r\n   at Microsoft.ClearScript.DocumentSettings.LoadDocument(Nullable`1 sourceInfo, String specifier, DocumentCategory category, DocumentContextCallback contextCallback)\r\n   at Microsoft.ClearScript.V8.V8ProxyHelpers.LoadModule(IntPtr pSourceDocumentInfo, String specifier, DocumentCategory category, UniqueDocumentInfo& documentInfo)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyManaged.LoadModule(IntPtr pSourceDocumentInfo, Ptr pSpecifier, Ptr pResourceName, Ptr pSourceMapUrl, UInt64& uniqueId, Boolean& isModule, Ptr pCode, IntPtr& pDocumentInfo)",
      "RemoteStackTraceString": null,
      "RemoteStackIndex": 0,
      "ExceptionMethod": "8\nLoadDocument\nClearScript.Core, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.DocumentLoader\nMicrosoft.ClearScript.Document LoadDocument(Microsoft.ClearScript.DocumentSettings, System.Nullable`1[Microsoft.ClearScript.DocumentInfo], System.String, Microsoft.ClearScript.DocumentCategory, Microsoft.ClearScript.DocumentContextCallback)",
      "HResult": -2146232799,
      "Source": "ClearScript.Core",
      "WatsonBuckets": null,
      "FileLoad_FileName": "lodash",
      "FileLoad_FusionLog": null
    },
    "HelpURL": null,
    "StackTraceString": "   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyNative.Invoke[T](Func`2 func)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8ContextProxyImpl.Execute(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ExecuteRaw(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ExecuteInternal(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass119_0.<Execute>b__0()\r\n   at Microsoft.ClearScript.ScriptEngine.ScriptInvokeInternal[T](Func`1 func)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass126_0`1.<ScriptInvoke>b__0()\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyManaged.InvokeHostAction(IntPtr pAction)",
    "RemoteStackTraceString": null,
    "RemoteStackIndex": 0,
    "ExceptionMethod": "8\nInvoke\nClearScript.V8, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.V8.SplitProxy.V8SplitProxyNative\nT Invoke[T](System.Func`2[Microsoft.ClearScript.V8.SplitProxy.IV8SplitProxyNative,T])",
    "HResult": -2146233079,
    "Source": "ClearScript.V8",
    "WatsonBuckets": null,
    "ScriptEngineName": "V8ScriptEngine",
    "ScriptErrorDetails": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.\n    at <anonymous>:0:0",
    "IsFatal": false,
    "ExecutionStarted": true
  },
  "HelpURL": null,
  "StackTraceString": "   at Microsoft.ClearScript.V8.SplitProxy.V8SplitProxyNative.Invoke(Action`1 action)\r\n   at Microsoft.ClearScript.V8.SplitProxy.V8ContextProxyImpl.InvokeWithLock(Action action)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.ScriptInvoke[T](Func`1 func)\r\n   at Microsoft.ClearScript.V8.V8ScriptEngine.Execute(UniqueDocumentInfo documentInfo, String code, Boolean evaluate)\r\n   at Microsoft.ClearScript.ScriptEngine.Execute(DocumentInfo documentInfo, String code)\r\n   at GCastV8.Engine.GCastEngine.Execute(String pathScript) in D:\\Projects\\others\\GCast\\GCastV8\\Engine\\GCastEngine.cs:line 248",
  "RemoteStackTraceString": null,
  "RemoteStackIndex": 0,
  "ExceptionMethod": "8\nInvoke\nClearScript.V8, Version=7.3.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35\nMicrosoft.ClearScript.V8.SplitProxy.V8SplitProxyNative\nVoid Invoke(System.Action`1[Microsoft.ClearScript.V8.SplitProxy.IV8SplitProxyNative])",
  "HResult": -2146233079,
  "Source": "ClearScript.V8",
  "WatsonBuckets": null,
  "ScriptEngineName": "V8ScriptEngine",
  "ScriptErrorDetails": "Error: Could not load file or assembly 'lodash' or one of its dependencies. The system cannot find the file specified.\n    at <anonymous>:0:0",
  "IsFatal": false,
  "ExecutionStarted": true
}
ClearScriptLib commented 1 year ago

Hi @stone89son,

Try setting DocumentSettings.FileNameExtensions to "js".

The default DocumentSettings object gets its list of file name extensions from the script engine, but since you're creating a new one, you need to set it yourself.

Another option is simply to modify the provided DocumentSettings object instead of creating a new one. Each script engine gets its own DocumentSettings object by default, so modifying it won't affect other engines.

Good luck!

stone89son commented 1 year ago

I set "js" to DocumentSettings.FileNameExtensions, but cannot working. same error.

ClearScriptLib commented 1 year ago

Hi @stone89son,

We can't reproduce your issue. If you can share a complete minimal sample that demonstrates the problem, we'll be happy to take a look.

Thanks!

stone89son commented 1 year ago

I create minimal sample in here. i send invitation access, you can commit source code, when you agree to access. git clone https://github.com/stone89son/test_modules_function_clearscript

ClearScriptLib commented 1 year ago

Thanks @stone89son. We've pushed a fix.

stone89son commented 1 year ago

git clone https://github.com/stone89son/test_modules_function_clearscript

I understand, because Document setting Standard Javascript, so only use Standard Javascript module, if CommonJS setting will can use common module but with require command to load module.

ClearScriptLib commented 1 year ago

Document setting Standard Javascript, so only use Standard Javascript module

That's correct. Theoretically, with some help from the host, a standard module could import a CommonJS module; in fact, Node.js supports that. ClearScript currently does not.