Library e_sqlite3 not found in VSTO4.0 .net4.8 app.

farzonl commented 10 months ago

SQLitePCLRaw handles many platforms and configurations. If you are having trouble, we typically need a LOT of information from you in order to help figure out what is going wrong.

The following questions are provided to help you see what kinds of information we need:

What version of SQLitePCLRaw are you using? 2.1.6 via SQLitePCLRaw.bundle_green

<PackageReference Include="SQLitePCLRaw.bundle_green">
      <Version>2.1.6</Version>
</PackageReference>

If you are using one of the SQLitePCLRaw bundle packages, which one?

SQLitePCLRaw.bundle_green

What platform are you running on? What operating system? Which version? What CPU? Windows 10 and 11 amd64

What target framework are you building for? .net 4.8.1 Are you on .NET Framework or the newer stuff (.NET Core, .NET 5+, etc)? .net framework

Are you using the command line, or an IDE? Which IDE? Which version of that IDE? ide

Is this problem something that just started happening? For example, were things working well for you and then you updated something and then the problem showed up? What changed?

no was happening on 2.1.0 What is the exact error message you are seeing when the problem happens?

Exception thrown: 'System.ComponentModel.Win32Exception' in SQLitePCLRaw.batteries_v2.dll
Exception thrown: 'System.Exception' in SQLitePCLRaw.batteries_v2.dll
2023-08-24 23:27:29.932 [AppCenter] ERROR: Failed to initialize sqlite3 provider.
System.Exception: Library e_sqlite3 not found
plat: win
suffix: DLL
possibilities (2):
    1) C:\Users\username\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\f0a1aaf0\009d039e_e66ed801\runtimes\win-x64\native\e_sqlite3.dll
    2) C:\Users\username\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\f0a1aaf0\009d039e_e66ed801\e_sqlite3.dll
win TryLoad: C:\Users\username\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\f0a1aaf0\009d039e_e66ed801\runtimes\win-x64\native\e_sqlite3.dll
thrown: System.ComponentModel.Win32Exception (0x80004005): The specified module could not be found

Are you using PackageReference or packages.config? PackageReference If you are using mobile platforms, are you on classic/legacy Xamarin or on .NET 6 and higher? N\A on windows Sometimes other packages using SQLitePCLRaw cause problems when they are mixed together. What other packages are you including in your project? I'm trying to use Microsoft.AppCenter, Microsoft.AppCenter.Analytics, & Microsoft.AppCenter.Crashes 5.0.2

More context here: https://github.com/microsoft/appcenter-sdk-dotnet/issues/1583#issuecomment-1692758915

How can we reproduce the problem you are seeing? Your issue will get attention much faster if you attach a minimal reproduction sample project.

Can't add a sample project, but pretty easy to reproduce create a skeleton vsto app add this class:

using Microsoft.AppCenter;
using Microsoft.AppCenter.Analytics;
using Microsoft.AppCenter.Crashes;

public static class AppAnalytics
    {
        public static void Init()
        {
                AppCenter.Start("PUT_VALID_STRING_HERE"
                      typeof(Analytics), typeof(Crashes));
        }
    }

then inside of ThisAddIn_Startup call AppAnalytics.Init();

private void ThisAddIn_Startup(object sender, System.EventArgs e)
        {
            AppAnalytics.Init();
        }

Work around I found was to copying the runtimes directory from C:\Users\username.nuget\packages\sqlitepclraw.lib.e_sqlite3\2.1.6\runtimes to C:\Users\username\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\f0a1aaf0\009d039e_e66ed801

That lets me keep developing locally but I can't ship with that.

It occurs to me that this is a similar workaround to https://github.com/natemcmaster/DotNetCorePlugins/issues/84 even though we are using different bundles.

This might be a dupe of https://github.com/ericsink/SQLitePCL.raw/issues/455 However that is an older .net version and I only have one instance of SQLitePCL.raw via AppCenter.

Or https://github.com/ericsink/SQLitePCL.raw/issues/343 But I'm not seeing any TypeInitializationExceptions

ericsink commented 10 months ago

Does the problem go away if you add a RuntimeIdentifier to your csproj ? Even if that's not a workable solution, it would be diagnostically interesting if it made a difference.

farzonl commented 10 months ago

No it doesn't. I got the idea on how to do what you suggested from this xunit test change: https://github.com/ericsink/SQLitePCL.raw/commit/1b2e50535cf3c3e4811197b72b7dc694715227f6?diff=unified#diff-16e5545b7bb35fe0eae10ea1944bb876be6622be8dcd39ca8b9d3fed18b9c6c6

Below is what I tried am I doing it wrong?

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|x86' ">
  <PlatformTarget>x86</PlatformTarget>
  <DebugSymbols>true</DebugSymbols>
  <DebugType>full</DebugType>
  <Optimize>false</Optimize>
  <RuntimeIdentifiers>win-x86;win-x64</RuntimeIdentifiers>
  <RuntimeIdentifier>win-x86</RuntimeIdentifier>
  <OutputPath>bin\x86\Debug\</OutputPath>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x86' ">
  <PlatformTarget>x86</PlatformTarget>
  <DebugType>pdbonly</DebugType>
  <Optimize>true</Optimize>
  <RuntimeIdentifiers>win-x86;win-x64</RuntimeIdentifiers>
  <RuntimeIdentifier>win-x86</RuntimeIdentifier>
  <OutputPath>bin\x86\Release\</OutputPath>
</PropertyGroup>
 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|x64' ">
  <PlatformTarget>x64</PlatformTarget>
  <DebugSymbols>true</DebugSymbols>
  <DebugType>full</DebugType>
  <Optimize>false</Optimize>
  <RuntimeIdentifiers>win-x86;win-x64</RuntimeIdentifiers>
  <RuntimeIdentifier>win-x64</RuntimeIdentifier>
  <OutputPath>bin\x64\Debug\</OutputPath>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x64' ">
  <PlatformTarget>x64</PlatformTarget>
  <DebugType>pdbonly</DebugType>
  <Optimize>true</Optimize>
  <RuntimeIdentifiers>win-x86;win-x64</RuntimeIdentifiers>
  <RuntimeIdentifier>win-x64</RuntimeIdentifier>
  <OutputPath>bin\x64\Release\</OutputPath>
</PropertyGroup>

farzonl commented 10 months ago

It did change one thing e_sqlite3.dll now shows up in the x64|x86 release|debug directories when before it did not:

Problem is the app isn't loading the dll from those directories.

It is trying to find it here

1) C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\b1e56d9e\009d039e_e66ed801\runtimes\win-x64\native\e_sqlite3.dll
    2) C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\b1e56d9e\009d039e_e66ed801\e_sqlite3.dll

This is like a temp directory that dlls get copied to at runtime: it is the directory that SQLitePCLRaw.core.DLL is at.

farzonl commented 10 months ago

Ok so did some digging. This looks very much like a vsto only problem. Most the documentation has been lost to time but archive.org really helped out: https://web.archive.org/web/20140424032444/http://blogs.msdn.com/b/vsod/archive/2008/11/01/office-customization-creates-loads-dll-from-temporary-folder.aspx

Relevant pieces:

"VSTO customization DLLs are always loaded from a temporary folder location and not from original location"

"The Shadow copy feature has been used in VSTO from the initial version and there is no way to disable it."

You can determine the original location of the assembly and the location from where the customization is being loaded by using below VB/C# code:
System.Reflection.Assembly.GetExecutingAssembly().CodeBase // Gives original assembly location
System.Reflection.Assembly.GetExecutingAssembly().Location // Gives temporary assembly location

So What I wanted to do is see if I could preload the e_sqlite dll before it was used by app center like so

 // Import the LoadLibrary function from kernel32.dll to load the DLL dynamically.
        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern IntPtr LoadLibrary(string dllToLoad);

        // Import the FreeLibrary function from kernel32.dll to unload the DLL.
        [DllImport("kernel32.dll")]
        public static extern bool FreeLibrary(IntPtr hModule);

        public static IntPtr Init()
        {
            IntPtr libPtr = IntPtr.Zero;
            try
            {
                AzureKeyVault kv = new AzureKeyVault();
                string binartDirectoryPath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().CodeBase);
                string e_sqlite3Path = Path.Combine(binartDirectoryPath, "e_sqlite3.dll");
                libPtr = LoadLibrary(new Uri(e_sqlite3Path).AbsolutePath);
                if(libPtr == IntPtr.Zero) { 
                    return IntPtr.Zero;
                }
                AppCenter.Start(kv.AnalyticsConnectionString(),
                      typeof(Analytics), typeof(Crashes));
            } catch(Exception ex)
            {
                Debug.WriteLine("Analytics failed to load. Turning them off.");
                Debug.WriteLine(ex.ToString());
                FreeLibrary(libPtr);
                libPtr = IntPtr.Zero;
            }
            return libPtr;
        }

Is there a way to use System.Reflection.Assembly.GetExecutingAssembly().CodeBase as a third dll lookup option for SQLitePCL.raw?

if you can do that then I'm sure all the VSTO specific bugs will go away.

farzonl commented 10 months ago

I'm think the fix should be here https://github.com/ericsink/SQLitePCL.raw/blob/9c66b4f618d9d6831e83c16a8036daea83df4ddb/src/common/nativelibrary_for_netstandard2.cs#L352C0-L373C0

We add | NativeLibrary.WHERE_CODEBASE https://github.com/ericsink/SQLitePCL.raw/blob/9c66b4f618d9d6831e83c16a8036daea83df4ddb/src/common/batteries_v2.cs#L105

and then after line 371 in nativelibrary_for_netstandard2.cs

if ((flags & WHERE_CODEBASE) != 0) {
               string binartDirectoryPath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().CodeBase);
               a.Add(Path.Combine(binartDirectoryPath, libname));
}

farzonl commented 10 months ago

@ericsink didn't see a no on this plan so put up a PR so you can see what it would look like: https://github.com/ericsink/SQLitePCL.raw/pull/553.

I don't have ms office at home, so don't sign off just yet I need to test it on my work machine tomorrow morning that has VSTO and office.

ericsink commented 10 months ago

Sorry, I got caught up in other stuff. Thanks for the detailed follow up work. I'll give this a look as soon as I can.

farzonl commented 10 months ago

Confirmed this works for VSTO apps:

Now we throw two exceptions for WHERE_RUNTIME_RID and WHERE_ADJACENT

'WINWORD.EXE' (CLR v4.0.30319: SomeApp.vsto|vstolocal): Loaded 'C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\c0d3eec6\a3ce3125_80dcd901\SQLitePCLRaw.batteries_v2.dll'. Symbols loaded.
'WINWORD.EXE' (CLR v4.0.30319: SomeApp.vsto|vstolocal): Loaded 'C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\df8755d4\acf53125_80dcd901\SQLitePCLRaw.core.dll'. Symbols loaded.
'WINWORD.EXE' (CLR v4.0.30319: SomeApp.vsto|vstolocal): Loaded 'C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\c747ec75\c1684f25_80dcd901\SQLitePCLRaw.provider.dynamic_cdecl.dll'. Symbols loaded.
Exception thrown: 'System.ComponentModel.Win32Exception' in SQLitePCLRaw.batteries_v2.dll
Exception thrown: 'System.ComponentModel.Win32Exception' in SQLitePCLRaw.batteries_v2.dll
'WINWORD.EXE' (CLR v4.0.30319: SomeApp.vsto|vstolocal): Loaded 'C:\Users\farzonlotfi\AppData\Local\assembly\dl3\GPDLTRHM.2DD\CW8H18OX.ZAX\43e2ac51\00dd8a2b_9ec7d701\System.Runtime.CompilerServices.Unsafe.dll'. Symbols loaded.

BUT there is no third exception for WHERE_CODEBASE.

net result im seeing my build in App center now:

I'm on VS 2022 and none of the ios\tvos stuff works for me. If whoever maybe reading this in the future is also stuck on this and you just need a bundle_green feel free to use my patch. ios_disable_patch.patch

Next thing you need to do is make %AppData%\Roaming\NuGet\NuGet.Config point to SQLitePCL.raw's nuget.config

you do that by changing package source to have a local nuget source

Before:

<packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>

After:

<packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="Local Packages Source" value="<SOME_PATH>\SQLitePCL.raw" />
  </packageSources>

ericsink commented 8 months ago

Just a quick comment here to say that I have not forgotten about this. Thanks for your patience. :-)

farzonl commented 7 months ago

fixed by https://github.com/ericsink/SQLitePCL.raw/pull/553

ericsink / SQLitePCL.raw

Library e_sqlite3 not found in VSTO4.0 .net4.8 app. #552