ericsink / SQLitePCL.raw

A Portable Class Library (PCL) for low-level (raw) access to SQLite
Apache License 2.0
512 stars 106 forks source link

Add support for linux-ppc64le target #525

Closed alhad-deshpande closed 1 year ago

alhad-deshpande commented 1 year ago

This PR consists of the changes to support ppc64le target.

janani66 commented 1 year ago

Adding some context to this request -- in .NET7 announced today by Microsoft, we now have support for the ppc64le ( Linux on IBM Power) . We would like to enable the .NET applications on ppc64le to talk to Database back ends and this PR is part of that work.

Appreciate consideration of this PR.

janani66 commented 1 year ago

Thank for merging this PR so quickly. What is the cadence of your releases - when is the next release scheduled for? Is there a publicly available daily build that we can use to test with?

ericsink commented 1 year ago

I plan to do a test release yet this week, and probably a full release next week.

ericsink commented 1 year ago

@janani66 Version 2.1.3-pre20221111222311 on nuget should contain your pull requests for linux-ppc64le support. Have you by chance had an opportunity to test that release and verify that it works?

Thanks!

alhad-deshpande commented 1 year ago

@ericsink We will test this today and confirm you once it works.

alhad-deshpande commented 1 year ago

Hi @ericsink Thanks for providing the nuget package version. I tried testing this on Intel and Power machines. It worked on Intel machine but I am getting below exception on Power machine.

Below are the steps that I used for testing this with IdentityServer4 repository.

  1. Clone dotnet/efcore repo and checkout release/7.0 branch.
  2. Add 2.1.3-pre20221111222311 in eng/Versions.props file
  3. Build efcore repo with command "./build.sh -c Release -pack"
  4. The .nupkg files gets generated in efcore/artifiacts/packages/Release/Shipping
  5. Clone IdentityServer4 repo (Done lot of dotnet version related changes and added some code fixes to work this with dotnet version 7.0.0)
  6. Add local path "efcore/artifiacts/packages/Release/Shipping" in NuGet.config of IdentityServer4 repo
  7. Change version "7.0.1" to 7.0.1 which is built in efcore in src/Directory.Build.targets file.
  8. ./build.sh

The IdentityServer4 build and tests works on Intel machine but on Power machine exception occurrs saying that e_sqlite3 DLL is not found. I am trying to check whats there in nuget package but Can you please have a look at the package whether it is there or not?

Below is the exception log: [xUnit.net 00:00:04.94] IdentityServer4.EntityFramework.IntegrationTests.Stores.ScopeStoreTests.GetAllResources_WhenAllResourcesRequested_ExpectAllResourcesIncludingHidden(options: DbContextOptions1 { ContextType = typeof(IdentityServer4.EntityFramework.DbContexts.ConfigurationDbContext), Extensions = [CoreOptionsExtension { ... }, InMemoryOptionsExtension { ... }], IsFrozen = False }) [FAIL] [[xUnit.net](http://xunit.net/) 00:00:05.00] IdentityServer4.EntityFramework.IntegrationTests.Stores.ScopeStoreTests.GetAllResources_WhenAllResourcesRequested_ExpectAllResourcesIncludingHidden(options: DbContextOptions1 { ContextType = typeof(IdentityServer4.EntityFramework.DbContexts.ConfigurationDbContext), Extensions = [SqliteOptionsExtension { ... }, CoreOptionsExtension { ... }], IsFrozen = False }) [FAIL] Failed IdentityServer4.EntityFramework.IntegrationTests.Stores.ScopeStoreTests.GetAllResources_WhenAllResourcesRequested_ExpectAllResourcesIncludingHidden(options: DbContextOptions`1 { ContextType = typeof(IdentityServer4.EntityFramework.DbContexts.ConfigurationDbContext), Extensions = [CoreOptionsExtension { ... }, InMemoryOptionsExtension { ... }], IsFrozen = False }) [1 ms] Error Message: System.TypeInitializationException : The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception. ---- System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation. -------- System.DllNotFoundException : e_sqlite3

ericsink commented 1 year ago

The file is present in the package.

$ unzip -l sqlitepclraw.lib.e_sqlite3.2.1.3-pre20221111222311.nupkg
Archive:  sqlitepclraw.lib.e_sqlite3.2.1.3-pre20221111222311.nupkg
  Length      Date    Time    Name
---------  ---------- -----   ----
      520  2022-11-11 22:25   _rels/.rels
     1102  2022-11-11 22:25   SQLitePCLRaw.lib.e_sqlite3.nuspec
...
  1382760  2022-11-11 22:20   runtimes/linux-ppc64le/native/libe_sqlite3.so
...
---------                     -------
 34417858                     38 files

Possibly useful (or not) diagnostic questions:

  1. Is linux-ppc64le the correct RID?
  2. Is that libe_sqlite3.so file valid? What are its dynamic dependencies, and are those deps present on your test system? If you write a minimal C program to load the library and get an entry point like int sqlite3_libversion_number(void);, does it work?
alhad-deshpande commented 1 year ago

Thanks @ericsink for the response. I will try sample program suggested by you and also trying other solutions to check what's going wrong on Power machine. I will get back to you once I get something.

alhad-deshpande commented 1 year ago

@ericsink I have figured out the issue. The problem was with the changes made in efcore repo before building on Intel machine. I had added previously only "2.1.3-pre20221111222311" in eng/Versions.props file but that is not sufficient. I have also updated versions for each instance of SQLitePCLRaw. in all .csproj files in efcore repo and also added below key in NuGet.config.

I have built efcore on Intel machine after making above changes and then used the generated nuget packages on Power machine. After using these packages it refers to correct version 2.1.3-pre20221111222311 for each package and installs it correctly and DLL Not Found exception got resolved. The build and test suites passed on Power.

@ericsink One question is - What can be the next steps to make this as a release version instead of pre release version and do we need to make any changes in efcore repo for that?

ericsink commented 1 year ago

I can't answer anything about what needs to happen for the efcore repo. You'd have to ask them.

One further thing you could verify would be to create a test project that uses efcore packages from nuget instead of using your own build of the efcore repo, but with an additional PackageReference to update SQLitePCLRaw to 2.1.3-pre20221111222311. It seems like such a test should work on Power, assuming that efcore itself did not need any changes.

Something like this:

  <ItemGroup>
    <PackageReference Include="microsoft.entityframeworkcore" Version="7.0.0" />
    <PackageReference Include="sqlitepclraw.bundle_e_sqlite3" Version="2.1.3-pre20221111222311" />
  </ItemGroup>

In any case, based on what you said above, it sounds like the ppc64le builds of e_sqlite3 are working okay, so I should be able to do a release version with no further changes to SQLitePCLRaw.

janani66 commented 1 year ago

@ericsink -- yes the ppc64le builds of e_sqlite3 ( 2.1.3-pre20221111222311) are working okay. Do let us know when you do a release version of this.

alhad-deshpande commented 1 year ago

@ericsink Kindly let us know once you do the release version with ppc64le support.

ericsink commented 1 year ago

@alhad-deshpande I have not forgotten. I have been waiting because there is a bug fix that seemed imminent, but I had hoped that would be done by now. I may need to just go ahead and do the release.

ericsink commented 1 year ago

@alhad-deshpande Version 2.1.3 (with ppc64le) has been published to nuget.org.

alhad-deshpande commented 1 year ago

Thanks @ericsink