search

dotnet / android

.NET for Android provides open-source bindings of the Android SDK for use with .NET managed languages such as C#
MIT License
1.92k stars 526 forks source link

Binding: Vision #5352

Closed jonpryor closed 5 months ago

jonpryor commented 3 years ago

The current binding infrastructure is "janky":

  1. "Somehow obtain" a .aar or .jar file to bind.
  2. Add it to a Binding project, and set the appropriate build action, e.g. @(LibraryProjectZip) or @(EmbeddedJar).
  3. Build it.

In practice, (1) and (3) can be incredibly annoying. (At least for @jonpryor, it's not always obvious where to obtain the files referenced by a Maven artifact, and (3) is full of peril.)

(3) will remain "out of scope" for this vision; fixing (3) involves ensuring that generator can always emit compilable output, and that we expand the Java constructs that generator supports. We should absolutely do this, but addressing these deficits involves finding the "less-than-ideal" constructs, filing issues for them, and fixing them.

Which leaves (1) and (2): can anything be done there?

Artifact Retrieval & Binding

Instead of a .aar or .jar file as being the "primary starting point", can we instead have a Maven artifact ID be the primary point?

Then, given a set of Maven artifact IDs:

  1. Specify the Maven IDs.
  2. Obtain the entire dependency graph for the maven artifacts.
  3. Find existing bindings for everything within (1).
  4. For any dependencies not already bound, bind them.

Specify the Maven IDs

How should the Maven IDs be specified? What's the "user interface"?

Obtain Maven Dependency Graph

I believe @moljac has code to do this, as part of Binderator: https://github.com/xamarin/XamarinComponents/tree/master/Util/Xamarin.AndroidBinderator

Find existing bindings

It appears that NuGet.org allows searching arbitrary package metadata. If we ensure that all new NuGet packages contain NuGet.org-searchable package metadata containing the Maven artifact ID & version, then it should be possible to find existing bindings.

@moljac is working on this.

Bind unbound packages

For any maven artifacts in the dependency graph which haven't been bound, presumably including the "root" dependencies, we should bind them.

Proposal: Item Group?

We can imagine using an MSBuild item group to specify the "root" maven IDs, either directly:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

or by using a file (file format? Gradle? JSON? XML?) to contain the information.

Implicit in this idea is that these are like @(PackageReference) item group: the containing .csproj can have as many of them as desired, which in turn means that the .csproj could emit "many" output assemblies (dozens?), one per bound library as well as the "intentioned" output assembly.

Additionally, how would @(TransformFile) (e.g. metadata.xml) work in such an environment? All @(TransformFile)s apply to all unbound artifacts?

Proposal: dotnet new?

Instead of using an item group, add a new tool, e.g.

dotnet new maven-binding --group-id androidx.car --artifact-id car --version 1.0.0-alpha7

This command would in turn emit a solution, with one .csproj project per unbound artifact. This "answers" the @(TransformFile) question: each project uses has its own set of @(TransformFile)s.

Less well defined is this: how does one update an existing "binding solution"? Changing the root artifact version will "ripple" across the entire solution; is it possible to update such a directory tree in-place? Or would we need to re-invoke the dotnet new maven-binding command into a new directory, and require the user manually copy over any required @(TransformFile)s & partial classes and such?

jonathanpeppers commented 3 years ago

is it possible to update such a directory tree in-place?

When working on #5348 I found you can do dotnet new maven-binding --force to write on top of existing directories and files. It does not delete any existing files when using --force.

jpobst commented 3 years ago

It sounds like this proposal is basically doing 2 things for the user: 1) Downloading the binary from Maven 2) Adding NuGet or Project dependencies automatically based on Maven metadata

#1 probably isn't too interesting. I don't think it used to be this way, but now you can just search on Maven and it provides an easy link to download the artifact: example.

We could probably even automate this without much effort in today's projects if we wanted to and resolve it at compile time:

Change:

<InputJar Include="okhttp-4.9.0.jar" />

to something like:

<MavenInputJar Group="com.squareup.okhttp3" Id="okhttp" Version="4.9.0" />

#2 is definitely more interesting. I'm not sure how much effort it normally takes.

Additional Considerations to Flesh Out

NuGet Metadata

From my reading of that link, I don't think NuGet supports searching arbitrary metadata. It seems to only support searching the official NuGet supported metadata. Thus we would need to decide which of those fields we would use.

Additions considerations:

Visual Studio UX

How extensible is the built-in VS template UI system? Can we prompt the user to enter Maven information that we can use? Is the template system robust enough that it can run our custom logic to query Maven and resolve dependencies in order to generate projects?

If we can only do this outside of VS, do we think users will find and use it?

Updates

Once a package has been bound and a new release is posted to Maven, what does the update process look like for updating our binding?

Cases to consider:

Publishing to NuGet

We need to understand what the target user is going to do with the binding: consume it locally or publish to NuGet.

If publishing to NuGet then there are a few additional issues like:

moljac commented 3 years ago

It sounds like this proposal is basically doing 2 things for the user:

  1. Downloading the binary from Maven

Trivial.

  1. Adding NuGet or Project dependencies automatically based on Maven metadata

Not only Maven. What if we have bindings? maven gives a sheet about nuget.org

From my reading of that link, I don't think NuGet supports searching arbitrary metadata. It seems to only support searching the official NuGet supported metadata. Thus we would need to decide which of those fields we would use.

I am testing client API for search

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.NuGet/NuGetClient.cs#L36-L73

NOTE: 

                                                    new SearchFilterType
                                                    {                                                       
                                                    }

I did not play with that more than 15 minutes.

How do we handle collisions? ie: More than one package says they are the package for com.squareup.okhttp3.okhttp.

dunno. Trying to have something - then I can make new steps...

Cases to consider:

Update the bound package version New version has a new dependency that is on NuGet New version has a new dependency that is not on NuGet New version removed a NuGet dependency New version removed a not-NuGet dependency

Writing xummary and pseudo-code. Coming soon

moljac commented 3 years ago

Obtain Maven Dependency Graph I believe @moljac has code to do this, as part of Binderator:
https://github.com/xamarin/XamarinComponents/tree/master/Util/Xamarin.AndroidBinderator

Nope. Side project. Will be easily embeddable. Or code copied with whatever changes needed (namespace changes etc)

Proposal: Item Group? We can imagine using an MSBuild item group to specify the "root" maven IDs, either directly:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

MavenArtifact would not normaly appear in .NET library or project, but Xamarin.Android. Thus we could add some convetion-based rules... Like checking for serialized data files (json, xml) with artifacts to bind...

If the ItemGroup is defined we could:

  1. check for serialized files and merge lists or
  2. ignore files

Why working with files? It would be easier to have such files as "interfaces" for design and testing.

Now more details:

We have 2 sources od ultimate divine truth (priority list):

  1. nuget.org
  2. maven repos (currently I work only with maven.google.com)

Pick this:

<MavenArtifact Include="androidx.car" ArtifactId="car" Version="1.0.0-alpha7" />

What I am trying to implement right now:

  1. query nuget.org if there is already something like "androidx car car" on nuget.org. Grab versions and metadata https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.NuGet/NuGetClient.cs#L43

  2. If there is NO such nuget package or the version is GREATER-THAN Version="1.0.0-alpha7" put in the List TODOs_ToBind

  3. Check dependencies from maven POM 

    
foreach (Artifact a in a.POM.Dependencies.Leaves()) // Leaves is GraphTheoretical term for flattening
{
    TODOs_ToBind = TODOs_ToBind.Concat
                                (
                                    ! a.CheckNugetOrgPackages()     // everything not on nuget.org must be bound 
                                                                    // .. or other BuildActions 
                                );
}

foreach (Artifact a in TODOs_ToBind)
{
    Process.Run("dotnet new maven-binding {a}");
}
'''
moljac commented 3 years ago 
Artifact a  = new Artifact().POM;

POM does not have stable name (yet) it is basically Maven 4.0.0 XSD schema transformed into C# classes

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/docs/maven-4.0.0.xsd

https://github.com/HolisticWare-Xamarin-Tools/HolisticWare.Xamarin.Tools.Bindings.XamarinAndroid.FassBinderMeister/blob/master/source/HolisticWare.Xamarin.Tools.Maven/Models.GeneratedFromXSD.Original/maven-4.0.0.cs

I used XML to from AndroidX and GPS-FB to generate C#, - BUT there were some fields missing. So last week I generated C# from maven XSD

moljac commented 3 years ago

My plan is 20201202 for next 2 weeks:

  1. create minimal API for step 3 in https://github.com/xamarin/xamarin-android/issues/5352#issuecomment-737339442

  2. pack/publish some HolisticWare packages with API

  3. create Cake scripts for testing that can be copied ANYWHERE (C# libs for MSBuild tasks or dotnet tools)

jpobst commented 3 years ago

An additional consideration from @Redth:

Maven is like NuGet, there can be many feeds, some of which may require authentication. So we may need to support specifying a URL, and possibly a username/password as well.

Redth commented 3 years ago

We should really back up a step here and sit down to discuss a few things:

  1. What are the different scenarios in which customers care about bindings
  2. What are the challenges in those scenarios
  3. What can we do to help with those challenges.
moljac commented 3 years ago
  1. What are the different scenarios in which customers care about bindings

In very few cases they do. Usually expect to be done by Microsoft or some partner. Basically wishes to have some SDK bindings are submitted as issues in (AndroidX/AndroidSupport, GPS-FB and XamarinComponents). Some are submitted as internal mails and very few (for me personally) over social media.

  1. What are the challenges in those scenarios
  1. Bindings knowledge and skills which influence productivity where most users give up.
  2. Bindings process 2.1 find/download artifact, 2.2 analyze artifact (dependencies and nuget packages)
  3. Test bindings (samples or unit tests)

Challenges I personally have:

  1. What can we do to help with those challenges.

Improving productivity will most likely increase abstraction (add one or more levels of APIs) that might drive some users away from bindings.

Needed APIs:

NOTE: More details to come.

Maven is like NuGet, there can be many feeds, some of which may require authentication. So we may need to support specifying a URL, and possibly a username/password as well.

I work with Google's maven right now, but I do plan to add others (mavenCentral and jcenter/Bintray).

I would not bother with private repos or repos that need authentication. There must be some reason artifacts are not public. We could add that later.

moljac commented 3 years ago

Maven API (extended)

20201208

Initial data for current version of binderator:

      {
        "groupId": "androidx.car",
        "artifactId": "car",
        "version": "1.0.0-alpha7",
        "nugetVersion": "1.0.0.3-alpha7",
        "nugetId": "Xamarin.AndroidX.Car.Car",
        "dependencyOnly": false
      },

https://github.com/xamarin/AndroidX/blob/master/config.json#L172-L179

NOTE:

For AndroidX and GPS-FB there is almost 200 artifacts to bind with growing number. So, I want to free my time so I can do other stuff including these improvements. Thus starting with binderator's config.json

Because it is manual process (currently) order of json data is important for me, because I can easily check/verify versions and names.

Artifact Versions

So with 20201208 there is extended info with versions (available for artifact):

{
  "groupId": "androidx.car",
  "artifactId": "car",
  "version": null,
  "idFullyQualified": "androidx.car.car",
  "versionTextual": "1.0.0-alpha5",
  "versions": null,
  "versionsTextual": [
    "1.0.0-alpha7",
    "1.0.0-alpha5",
    "1.0.0-alpha4",
    "1.0.0-alpha3",
    "1.0.0-alpha1"
  ],
  "projectObjectModelTextual": null,
  "projectObjectModel": null,
  "dependencies": null
}

This will provide opportunity to verify version given for bindings ("1.0.0-alpha6" would fail) and to make plan for future bindings or even automate (generate next config, based on current)

Cache

WIP - deserializing locally dumped Maven repo data - if not grab it.

ProjectObjectModel POM

Currently POM is grabbed and deserialized.

WIP on extracting important data like dependecies:

{
  "groupId": "androidx.car",
  "artifactId": "car",
  "version": null,
  "idFullyQualified": "androidx.car.car",
  "versionTextual": "1.0.0-alpha5",
  "versions": null,
  "versionsTextual": null,
  "projectObjectModelTextual": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\" xmlns=\"http://maven.apache.org/POM/4.0.0\"\n    xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n  <modelVersion>4.0.0</modelVersion>\n  <groupId>androidx.car</groupId>\n  <artifactId>car</artifactId>\n  <version>1.0.0-alpha5</version>\n  <packaging>aar</packaging>\n  <name>Android Car Support UI</name>\n  <description>Android Car Support UI</description>\n  <url>http://developer.android.com/tools/extras/support-library.html</url>\n  <inceptionYear>2017</inceptionYear>\n  <licenses>\n    <license>\n      <name>The Apache Software License, Version 2.0</name>\n      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>\n      <distribution>repo</distribution>\n    </license>\n  </licenses>\n  <developers>\n    <developer>\n      <name>The Android Open Source Project</name>\n    </developer>\n  </developers>\n  <scm>\n    <connection>scm:git:https://android.googlesource.com/platform/frameworks/support</connection>\n    <url>http://source.android.com</url>\n  </scm>\n  <dependencies>\n    <dependency>\n      <groupId>com.google.android.material</groupId>\n      <artifactId>material</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n      <exclusions>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.appcompat</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.annotation</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.transition</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.recyclerview</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.legacy</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.fragment</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.cardview</groupId>\n        </exclusion>\n        <exclusion>\n          <artifactId>*</artifactId>\n          <groupId>androidx.core</groupId>\n        </exclusion>\n      </exclusions>\n    </dependency>\n    <dependency>\n      <groupId>androidx.appcompat</groupId>\n      <artifactId>appcompat</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.cardview</groupId>\n      <artifactId>cardview</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.annotation</groupId>\n      <artifactId>annotation</artifactId>\n      <version>1.0.0-rc01</version>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.legacy</groupId>\n      <artifactId>legacy-support-v4</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.recyclerview</groupId>\n      <artifactId>recyclerview</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.gridlayout</groupId>\n      <artifactId>gridlayout</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.preference</groupId>\n      <artifactId>preference</artifactId>\n      <version>1.0.0-rc01</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n    <dependency>\n      <groupId>androidx.constraintlayout</groupId>\n      <artifactId>constraintlayout</artifactId>\n      <version>1.1.0</version>\n      <type>aar</type>\n      <scope>compile</scope>\n    </dependency>\n  </dependencies>\n</project>\n",
  "projectObjectModel": {
    "modelVersion": "4.0.0",
    "parent": null,
    "packaging": "aar",
    "name": "Android Car Support UI",
    "description": "Android Car Support UI",
    "url": "http://developer.android.com/tools/extras/support-library.html",
    "inceptionYear": "2017",
    "organization": null,
    "licenses": [
      {
        "name": "The Apache Software License, Version 2.0",
        "url": "http://www.apache.org/licenses/LICENSE-2.0.txt",
        "distribution": "repo",
        "comments": null
      }
    ],
    "developers": [
      {
        "id": null,
        "name": "The Android Open Source Project",
        "email": null,
        "url": null,
        "organization": null,
        "organizationUrl": null,
        "roles": null,
        "timezone": null,
        "properties": null
      }
    ],
    "contributors": null,
    "mailingLists": null,
    "prerequisites": null,
    "modules": null,
    "scm": {
      "connection": "scm:git:https://android.googlesource.com/platform/frameworks/support",
      "developerConnection": null,
      "tag": "HEAD",
      "url": "http://source.android.com",
      "childscmconnectioninheritappendpath": null,
      "childscmdeveloperConnectioninheritappendpath": null,
      "childscmurlinheritappendpath": null
    },
    "issueManagement": null,
    "ciManagement": null,
    "distributionManagement": null,
    "properties": null,
    "dependencyManagement": null,
    "repositories": null,
    "pluginRepositories": null,
    "build": null,
    "reports": null,
    "reporting": null,
    "profiles": null,
    "childprojecturlinheritappendpath": null,
    "groupId": "androidx.car",
    "artifactId": "car",
    "version": "1.0.0-alpha5",
    "dependencies": [
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": [
          {
            "groupId": "androidx.appcompat",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.annotation",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.transition",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.recyclerview",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.legacy",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.fragment",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.cardview",
            "artifactId": "*"
          },
          {
            "groupId": "androidx.core",
            "artifactId": "*"
          }
        ],
        "optional": null,
        "groupId": "com.google.android.material",
        "artifactId": "material",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.appcompat",
        "artifactId": "appcompat",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.cardview",
        "artifactId": "cardview",
        "version": "1.0.0-rc01"
      },
      {
        "type": "jar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.annotation",
        "artifactId": "annotation",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.legacy",
        "artifactId": "legacy-support-v4",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.recyclerview",
        "artifactId": "recyclerview",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.gridlayout",
        "artifactId": "gridlayout",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.preference",
        "artifactId": "preference",
        "version": "1.0.0-rc01"
      },
      {
        "type": "aar",
        "classifier": null,
        "scope": "compile",
        "systemPath": null,
        "exclusions": null,
        "optional": null,
        "groupId": "androidx.constraintlayout",
        "artifactId": "constraintlayout",
        "version": "1.1.0"
      }
    ]
  },
  "dependencies": null
}

Extras

moljac commented 3 years ago

NuGet API

20201208

Currently there are 3 Nuget Client API use cases:

Search is very limited regarding filtering and customizations and searches using Contains() API. So, API with Predicate had to be added and

Furthermore Search does not return DependecySets to filter out based on TargetFramework, so for each nuget GetMetadataAsync(...) must be called. I open an issue in Nuget Client API repo. The idea was not to surface NuGetClient classes, but for now I had to.

Search

So, the closest API in few days that returns the same results (3 results for "androidx+car" search ) is:

            ArtifactBindingNuget abnd = new ArtifactBindingNuget("androidx.car", "car")
            {
                NuGetId = "Xamarin.AndroidX.Car.Car"
            };

            IEnumerable<global::NuGet.Protocol.Core.Types.IPackageSearchMetadata> result = null;

            result = abnd.SearchPackagesByKeywordWithFilterAsync
                            (
                                abnd.NuGetId,
                                // null,
                                new global::NuGet.Protocol.Core.Types.SearchFilter
                                                                        (
                                                                            includePrerelease: true
                                                                        ),
                                skip: 0,
                                take: 100,
                                // default (for null) predicates:
                                // 
                                // custom predicate:
                                psm =>
                                {
                                    return
                                    (
                                        (
                                            psm.Title.ToLower().Contains("androidx")
                                            &&
                                            psm.Title.ToLower().Contains("car")
                                        )
                                        &&
                                        (
                                            psm.Description.ToLower().Contains("car")
                                            ||
                                            psm.Description.ToLower().Contains("androidx.car")
                                        )
                                    );
                                }
                            )
                            .Result;

And it returns serialized:

{
  "nuGetId": "Xamarin.AndroidX.Car.Car",
  "nuGetPackagesSearchResults": [
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - car",
      "downloadCount": 539,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.car.car/1.0.0.3-alpha7/icon",
      "identity": {
        "id": "Xamarin.AndroidX.Car.Car",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 3,
          "isSemVer2": false,
          "originalVersion": "1.0.0.3-alpha7",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [
            "alpha7"
          ],
          "release": "alpha7",
          "isPrerelease": true,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 3,
            "majorRevision": 0,
            "minorRevision": 3
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.Car.Car/1.0.0.3-alpha7/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - car",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, car",
      "title": "Xamarin AndroidX - car",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    },
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - cardview",
      "downloadCount": 1080465,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.cardview/1.0.0.5/icon",
      "identity": {
        "id": "Xamarin.AndroidX.CardView",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 5,
          "isSemVer2": false,
          "originalVersion": "1.0.0.5",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [],
          "release": "",
          "isPrerelease": false,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 5,
            "majorRevision": 0,
            "minorRevision": 5
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.CardView/1.0.0.5/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - cardview",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, cardview",
      "title": "Xamarin AndroidX - cardview",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    },
    {
      "authors": "Microsoft",
      "dependencySets": [],
      "description": "Xamarin.Android bindings for AndroidX - car-cluster",
      "downloadCount": 549,
      "iconUrl": "https://api.nuget.org/v3-flatcontainer/xamarin.androidx.car.cluster/1.0.0.3-alpha5/icon",
      "identity": {
        "id": "Xamarin.AndroidX.Car.Cluster",
        "hasVersion": true,
        "version": {
          "isLegacyVersion": true,
          "revision": 3,
          "isSemVer2": false,
          "originalVersion": "1.0.0.3-alpha5",
          "major": 1,
          "minor": 0,
          "patch": 0,
          "releaseLabels": [
            "alpha5"
          ],
          "release": "alpha5",
          "isPrerelease": true,
          "hasMetadata": false,
          "metadata": "",
          "version": {
            "major": 1,
            "minor": 0,
            "build": 0,
            "revision": 3,
            "majorRevision": 0,
            "minorRevision": 3
          }
        }
      },
      "licenseUrl": "https://www.nuget.org/packages/Xamarin.AndroidX.Car.Cluster/1.0.0.3-alpha5/license",
      "owners": null,
      "projectUrl": "https://go.microsoft.com/fwlink/?linkid=2113238",
      "published": null,
      "reportAbuseUrl": null,
      "packageDetailsUrl": null,
      "requireLicenseAcceptance": false,
      "summary": "Xamarin.Android bindings for AndroidX - car-cluster",
      "tags": "Xamarin, AndroidX, Xamarin.AndroidX, Support, Google, car-cluster",
      "title": "Xamarin AndroidX - car-cluster",
      "prefixReserved": true,
      "licenseMetadata": null,
      "vulnerabilities": null,
      "isListed": true,
      "packageReader": null
    }
  ]
}

Extras

Redth commented 3 years ago

I did a spike on a lot of this some time ago: https://github.com/Redth/Xamarin.Binding.Helpers

The idea was to use a .gradle file to specify the dependencies for the binding and let gradle/maven resolve the dependency graph for you, since that's what it's good at. With a small plugin added to the .gradle file, I was able to serialize all of the dependency graph from gradle in a way that msbuild tasks could parse it out and use it to obtain the dependency tree info (maven artifact group, id, version, binaries, etc). This could all be completely automated such that a gradle config is created for the top level maven artifacts specified by the developer and gradle can be downloaded and invoked on behalf of the user to get this info. The bonus here is things like maven feed authentication do not need re-inventing or engineering.

Once the dependency graph was known, all of the actual aar/jar binaries could be downloaded and included as appropriate items in the binding project. However, as mentioned in this issue several times, there's already lots of nuget packages out there that ship these artifacts in binding projects and would conflict during builds. Often these packages are desirable instead of just bundling the artifacts since their binding c# api surface may be required.

I did some work in the same repo around matching up NuGet packages. The idea of having metadata in nuget packages to match up their maven artifacts is great.... except... we'll likely never be able to get the entire ecosystem updated to this model. If that's an assumption then this mechanism would only be reliable for packages we control, or otherwise well known community packages. If we assume that, we can pretty much identify that set of packages already and map out some rules for the artifacts that packages map.

This is exactly what I did here: https://github.com/Redth/Xamarin.Binding.Helpers/blob/main/Xamarin.Binding.Helpers/NuGetResolvers/AndroidXMavenNugetResolver.cs

There are others for Firebase, GPS, and other well known package: https://github.com/Redth/Xamarin.Binding.Helpers/blob/main/Xamarin.Binding.Helpers/NuGetResolvers/KnownMavenNugetResolver.cs#L12-L98

It's really not that bad of a set of rules/mappings to have/maintain perhaps in addition to new packages being able to publish maven artifact related metadata (and fallback to these mappings).

The idea was that these rules would apply, and/or you could specify explicit mappings yourself if you know they exist, like this:

<ItemGroup>
  <MavenNuGetMapping MavenGroupId="com.google.zxing" MavenArtifactId="core" MavenVersion="3.3.3" NuGetPackageId="Xamarin.Google.ZXing.Core" NuGetVersion="3.3.3" /></ItemGroup>

So, for this whole library you would basically reference and android studio / gradle project in your xamarin binding project like this:

<ItemGroup>
  <AndroidStudioProject Include="/path/to/project" Module="mylibrary" GenerateBinding="True" />
</ItemGroup>

This would even go build the native project for you and include the module's binary (aar) as a so that it would generate bindings (or not if you disabled that option). It would link in the entire dependency tree of maven artifacts based on the android project's declared compile dependencies, but it would substitute those for NuGet packages according to the resolvers or explicit mapping override info you specified.

This isn't a perfect approach by any means, and I could see taking it a step further to not require the android studio project at all, but this is a good start to look at. Generally you wouldn't need to do this

jonpryor commented 2 years ago

It looks like gradlew -I INIT-SCRIPT can be specified multiple times: https://github.com/gradle/gradle/blob/edb2cc76c1d398633843303a42ad683db0fe8393/subprojects/core/src/main/java/org/gradle/initialization/StartParameterBuildOptions.java#L238-L251

Consequently, it should be possible to distribute a .gradle file with "something" (Android SDK, NuGet package) and pass this file to gradlew when invoking it. (Perhaps via a @(AndroidGradleInitScript) item group?)

Assuming this works, it means we can reliably add a Gradle task to retrieve the artifacts of the Gradle build, so that we can automagically determine paths to built artifacts + dependencies.

jpobst commented 2 years ago

I've been giving this some thought recently, and maybe we should do less automatically for the user (since that's where the dragons live), and focus first on helping the user ensure they are creating correct bindings.

TL:DR

We will initially focus on tackling two pain points of binding from Maven:

Moved proposal details to https://github.com/xamarin/xamarin-android/issues/4528.

jpobst commented 5 months ago

We will initially focus on tackling two pain points of binding from Maven:

  • Acquiring the .jar/.aar and the related .pom from Maven
  • Using the .pom to verify that required Java dependencies are being fulfilled

These features have been implemented for .NET 9 in the following PRs:

I think the rest of this issue is conjectural and not really actionable. Any additional steps we want to consider taking should probably be proposed and fleshed out in new issue(s).