jpobst / Prototype.Android.MavenBindings

Prototype of MSBuild tasks to facilitate binding of Java libraries from Maven repositories
MIT License
5 stars 3 forks source link

Automatically include transitive Java dependencies #10

Open vyacheslav-volkov opened 1 year ago

vyacheslav-volkov commented 1 year ago

First of all thanks for this project, I really like the idea of ​​slim bindings, I've been using my own slim bindings for years and it's much better than full C# binding for all Java types. Now, I'm using https://github.com/kezong/fat-aar-android to include all .aar files to a native library.

I tried to use this library to add com.walletconnect:sign:2.18.0, com.walletconnect:android-core:1.20.0 and in my opinion it is completely unusable in its current state. I have a native project written in Kotlin which uses these libs and I want to include all .aar dependencies in a .NET project to get my native project working. I get a lot of messages on dependencies and it is almost impossible to solve them all, moreover, if they update their library, I will have to repeat all the steps again to find all the dependencies and update them. Try including these libraries and you will understand me.

My suggestions: It would be nice to automatically resolve all dependencies and add all .aar.jar files without manually describing them in .csproj, something like this (the same way like fat aar includes them):

<AndroidMavenLibrary Include="com.walletconnect:android-core" Version="1.20.0"/><!-- automatically include all dependencies except critical like  Xamarin.AndroidX.., Xamarin.Kotlin... etc-->
<AndroidMavenLibrary Include="com.walletconnect:sign" Version="2.18.0"/>

and only show nuget package errors for critical packages like Xamarin.AndroidX.., Xamarin.Kotlin.., etc. For example, I don't use Square.OkIO at all from C# code, but it will show me an error that I need to install the nuget package instead of just including the raw .aar file. Perhaps you should add some kind of mode that will simply download and include all .aar files in the project, without matching them against all nuget packages, only the critical ones.

Here is the list of libs that I added and gave up because the number of errors is growing exponentially:

        <AndroidMavenLibrary Include="com.walletconnect:android-core" Version="1.20.0"/>
        <AndroidMavenLibrary Include="com.walletconnect:foundation" Version="1.10.0"/>
        <AndroidMavenLibrary Include="com.squareup.sqldelight:android-driver" Version="1.5.4"/>
        <AndroidMavenLibrary Include="com.squareup.sqldelight:coroutines-extensions-jvm" Version="1.5.4"/>
        <AndroidMavenLibrary Include="net.zetetic:android-database-sqlcipher" Version="4.5.3"/>
        <AndroidMavenLibrary Include="androidx.security:security-crypto-ktx" Version="1.1.0-alpha04" Repository="Google"/>
        <AndroidMavenLibrary Include="io.insert-koin:koin-android" Version="3.4.2"/>
        <AndroidMavenLibrary Include="com.jakewharton.timber:timber" Version="5.0.1"/>
        <AndroidMavenLibrary Include="com.github.WalletConnect.Scarlet:lifecycle-android" Version="1.0.0" Repository="https://maven.scijava.org/content/repositories/public/"/>
        <AndroidMavenLibrary Include="org.web3j:crypto" Version="4.9.5"/>
        <AndroidMavenLibrary Include="com.github.komputing.kethereum:bip39" Version="0.85.7" Repository="https://jitpack.io"/>
        <AndroidMavenLibrary Include="com.github.komputing.kethereum:bip39_wordlist_en" Version="0.85.7" Repository="https://jitpack.io"/>
        <AndroidMavenLibrary Include="com.github.komputing.kethereum:bip32" Version="0.85.7" Repository="https://jitpack.io"/>
        <AndroidMavenLibrary Include="com.github.komputing.kethereum:crypto_impl_spongycastle" Version="0.85.7" Repository="https://jitpack.io"/>
        <AndroidMavenLibrary Include="com.github.komputing.kethereum:model" Version="0.85.7" Repository="https://jitpack.io"/>
        <AndroidMavenLibrary Include="com.squareup.retrofit2:retrofit" Version="2.9.0"/>
        <AndroidMavenLibrary Include="com.squareup.retrofit2:converter-moshi" Version="2.9.0"/>
        <AndroidMavenLibrary Include="com.squareup.sqldelight:runtime-jvm" Version="1.5.4"/>
        <AndroidMavenLibrary Include="com.getkeepsafe.relinker:relinker" Version="1.4.5"/>

        <PackageReference Include="Xamarin.AndroidX.Sqlite" Version="2.3.1.3" />

        <PackageReference Include="Xamarin.Kotlin.StdLib" Version="1.9.0.1" />
        <PackageReference Include="Xamarin.Kotlin.StdLib.Common" Version="1.9.0.1" />
        <PackageReference Include="Xamarin.Kotlin.StdLib.Jdk8" Version="1.9.0.1"/>
        <PackageReference Include="Xamarin.KotlinX.Coroutines.Core" Version="1.7.3"/>

here is the list of errors:


0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.WalletConnect.Scarlet:scarlet' version '1.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.WalletConnect.Scarlet:websocket-okhttp' version '1.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.WalletConnect.Scarlet:stream-adapter-coroutines' version '1.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.WalletConnect.Scarlet:message-adapter-moshi' version '1.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Warning  : Could not determine needed version of Maven dependency 'com.squareup.okhttp3:okhttp' (possibly due to not understanding a parent POM). Validation of this dependency will be skipped, but it still needs to be fulfilled.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Warning  : Could not determine needed version of Maven dependency 'com.squareup.okhttp3:logging-interceptor' (possibly due to not understanding a parent POM). Validation of this dependency will be skipped, but it still needs to be fulfilled.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.squareup.moshi:moshi-adapters' version '1.14.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.squareup.moshi:moshi-kotlin' version '1.14.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.bouncycastle:bcprov-jdk15on' version '1.70' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.multiformats:java-multibase' version '1.1.1' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'io.insert-koin:koin-core-jvm' version '3.3.2' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'jakarta.ws.rs:jakarta.ws.rs-api' version '3.1.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.sqlite:sqlite-framework' version '2.1.0' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Sqlite.Framework' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm' version '1.5.2' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.KotlinX.Coroutines.Core.Jvm' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.security:security-crypto' version '1.1.0-alpha04' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Security.SecurityCrypto' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'io.insert-koin:koin-core' version '3.4.2' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.appcompat:appcompat' version '1.6.1' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.AppCompat' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.activity:activity-ktx' version '1.6.1' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Activity.Ktx' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.fragment:fragment-ktx' version '1.5.7' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Fragment.Ktx' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.lifecycle:lifecycle-viewmodel-ktx' version '2.6.1' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Lifecycle.ViewModel.Ktx' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.lifecycle:lifecycle-common-java8' version '2.6.1' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.Lifecycle.Common.Java8' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.jetbrains:annotations' version '20.1.0' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.Jetbrains.Annotations' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.WalletConnect.Scarlet:scarlet' version '1.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'io.reactivex.rxjava2:rxjava' version '2.1.16' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.Android.ReactiveX.RxJava' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'io.reactivex.rxjava2:rxkotlin' version '2.2.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'androidx.appcompat:appcompat' version '1.2.0' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.AndroidX.AppCompat' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.jetbrains.kotlin:kotlin-stdlib-jdk7' version '1.5.30' is not satisfied. Microsoft maintains the NuGet package 'Xamarin.Kotlin.StdLib.Jdk7' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.web3j:abi' version '4.9.5' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.web3j:rlp' version '4.9.5' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.web3j:utils' version '4.9.5' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'org.slf4j:slf4j-api' version '1.7.30' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.fasterxml.jackson.core:jackson-databind' version '2.13.3' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:extensions_kotlin' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto_api' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing:khex' version '1.1.2' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.khash:sha256-jvm' version '1.1.1' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto_api' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:extensions_kotlin' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing:kbip44' version '0.5' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing:kbase58' version '0.4' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.khash:sha256-jvm' version '1.1.1' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.khash:ripemd160-jvm' version '1.1.1' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.madgag.spongycastle:prov' version '1.58.0.0' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto_api' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:crypto_impl_java_provider' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing:khex' version '1.1.2' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.github.komputing.kethereum:extensions_kotlin' version '0.85.7' is not satisfied.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.squareup.okhttp3:okhttp' version '3.14.9' is not satisfied. Microsoft maintains the NuGet package 'Square.OkHttp3' that could fulfill this dependency.
0>XamPrototype.Android.MavenBinding.Tasks.targets(28,5): Error  : Maven dependency 'com.squareup.moshi:moshi' version '1.8.0' is not satisfied.
jpobst commented 1 year ago

Thanks for trying out the prototype and providing feedback!

I've definitely received feedback that having to add a line for all transitive dependencies is a pain. I'd like to do something to ease that pain, but I haven't figured out what can safely be done for it.

Let's take your Square.OkIO example and automatically add okio-3.5.0.jar to your project. But perhaps another NuGet package you are using does use Square.OkIO C# bindings and has a transitive reference to the Square.OkIO version 2.10.0 NuGet package.

Now we are adding both okio-3.5.0.jar and okio-2.10.0.jar to the application, which won't compile. Theoretically we could arbitrarily pick one, like the newest 3.5.0, but that .jar may have breaking changes and it won't work with the Square.OkIO 2.10.0 C# bindings.

These are the kind of issues that prevent going down the fully automatic path. So, as a first step, we've started with forcing the user to create projects with "proper" dependency chains.

One future idea I was thinking about that may help alleviate the pain without causing other issues would be a dotnet global tool that would generate the initial full project(s) with all the transitive dependencies specified for you.

Example:

maven-bind com.walletconnect:android-core:1.20.0 WalletConnect.csproj

This would generate a csproj that has all the appropriate <AndroidMavenLibrary> or <PackageReference> lines for all transitive dependencies. This would get it done the first time (which is likely to be the most painful), but you would have to make any updates as needed when you move to a different version. Or I guess you could regenerate the whole thing from scratch for a new version.

vyacheslav-volkov commented 1 year ago

Thanks for the response! I still think you should add the ability to load all transitive dependencies and also the ability to add some custom ignore rules, it will make our life easier. I'm pretty sure that if a developer uses slim bindings, he will know what he's doing and how to work with it:

<AndroidMavenLibrary Include="com.walletconnect:sign" Version="2.18.0" Transitive="True">
    <AndroidMavenLibraryExclude Include="org.jetbrains.*"/> <!-- not real syntax just for example -->
</AndroidMavenLibrary>

Now we are adding both okio-3.5.0.jar and okio-2.10.0.jar to the application, which won't compile. Theoretically we could arbitrarily pick one, like the newest 3.5.0, but that .jar may have breaking changes and it won't work with the Square.OkIO 2.10.0 C# bindings. These are the kind of issues that prevent going down the fully automatic path. So, as a first step, we've started with forcing the user to create projects with "proper" dependency chains.

If the developer includes conflicting bindings, as you correctly noted, the app won't compile, and the result will be the same as a validation error from the library. I understand that you want to somehow fix this automatically, but no one except the developer can fix it and both errors are equivalent, as a result the project will not compile. If we could manage all the Maven libraries the same way we do in Android studio, that would be amazing.