curiosity-ai / catalyst

🚀 Catalyst is a C# Natural Language Processing library built for speed. Inspired by spaCy's design, it brings pre-trained models, out-of-the box support for training word and document embeddings, and flexible entity recognition models.
MIT License
742 stars 75 forks source link

Could not load type 'Catalyst.ISpan' #74

Closed fbrier closed 2 years ago

fbrier commented 2 years ago

Describe the bug It appears that the Catalyst.Spacy.Pipeline.SyncBack() method cannot load Catalyst.ISpan. Since Catalyst.Spacy has Catalyst as a dependent NuGet package, it is not clear why it cannot load the class. Catalyst.ISpan make reference to System.ReadOnlySpan in System.Memory, whose NuGet package is referenced from Catalyst.

System.TypeLoadException : Could not load type 'Catalyst.ISpan' from assembly 'Catalyst, Version=1.0.31087.0, Culture=neutral, PublicKeyToken=null'.
   at Catalyst.Spacy.Pipeline.SyncBack(Object s_doc, Document document)
   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid3[T0,T1,T2](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   at Catalyst.Spacy.Pipeline.ProcessSingle(Document document)
   at OpenQuestionModel.Services.QueryService.GenerateQuery(String text) in D:\Projects\OpenQuestionLibrary\OpenQuestionModel\Services\QueryService.cs:line 47
   at OpenQuestionModel.Services.AnalyzerService.Analyze(String question) in D:\Projects\OpenQuestionLibrary\OpenQuestionModel\Services\AnalyzerService.cs:line 32
   at OpenQuestionModel.Test.AnalyzerServiceTest.TestAnalyzerService() in D:\Projects\OpenQuestionLibrary\OpenQuestionModel.Test\AnalyzerServiceTest.cs:line 32

Python version: 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
Spacy version: 3.4
https://github.com/explosion/spacy-models/releases/download/xx_ent_wiki_sm-3.4.0/xx_ent_wiki_sm-3.4.0.tar.gz#egg=xx_ent_wiki_sm==3.4.0
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.0/en_core_web_sm-3.4.0.tar.gz#egg=en_core_web_sm==3.4.0

If anyone else has bumped their head on this problem, please let me know the workaround. Thank you.

To Reproduce I have not created a simplified test case yet that reproduces the problem. The current test integrates a number of services including the initialization of Catalyst using Ninject.

Expected behavior Execute the SyncBack() call without error.

Additional context The Catalyst.Spacy.Test.Program executes without error, although in Debug mode it fails. The test makes the same call, Catalyst.Spacy.Pipeline.ProcessSingle(doc), and does not fail.

There is an unresolved issue in DotNet related to dynamic binding, which is used in the SyncBack() method..

fbrier commented 2 years ago

Below is a simplified nunit test that fails with the above error.

using Catalyst;
using Mosaik.Core;

namespace OpenQuestionModel.Test
{
    public class SimpleProcessSingleTest
    {
        private Spacy.PythonLock? _pythonLock;
        private Spacy.Pipeline? _pipeline;

        [SetUp]
        public void Setup()
        {
            _pipeline = CreatePipeline().Result;
        }

        [TearDown]
        public void TearDown()
        {
            _pipeline?.Dispose();
            _pythonLock?.Dispose();
        }

        [Test]
        public void TestProcessSingle()
        {
            var doc = new Document("founder Amazon", Language.English);
            // Fails here.
            _pipeline?.ProcessSingle(doc);
        }

        private async Task<Spacy.Pipeline> CreatePipeline()
        {
            Catalyst.Models.English.Register();
            _pythonLock = await Spacy.Initialize(Spacy.ModelSize.Small, Language.Any, Language.English);
            var pipeline = Spacy.For(Spacy.ModelSize.Small, Language.English);
            return pipeline;
        }
    }
}
fbrier commented 2 years ago

This appears to have been fixed.