GoogleCloudPlatform / artifact-registry-maven-tools

Apache License 2.0
22 stars 23 forks source link

Use artifact registry for Gradle Plugins #66

Open raminqaf opened 2 years ago

raminqaf commented 2 years ago

I use artifact registry to publish my project artifacts including costume Gradle plugins I developed for my project. However, whenever I want to use my published plugins in other projects, gradle fails to recognize the registry URL. In my settings.gradle.kts file you can see:

pluginManagement {
    repositories {
        gradlePluginPortal()
        mavenCentral()
        maven(url = "artifactregistry://us-central1-maven.pkg.dev/my-registry/my-repository-for-gradle-plugins")
    }
}

I have hoped that my plugins get added after I define them in build.gradle.kts like this:

plugins {
    java
    `java-test-fixtures`
    idea
    id("com.test.gradle.myplugin.custom.plugin") version "0.0.1"
    id("com.google.cloud.artifactregistry.gradle-plugin") version "2.1.4"
}
// Other part of build.gradle.kts
// ....
subprojects {
    apply(plugin = "java")
    apply(plugin = "java-test-fixtures")
    apply(plugin = "idea")
    apply(plugin = "com.test.gradle.myplugin.custom.plugin")
    apply(plugin = "com.google.cloud.artifactregistry.gradle-plugin")

    repositories {
        mavenCentral()
        maven(url = "artifactregistry://us-central1-maven.pkg.dev/my-registry/my-repository-for-dependencies")
    }
}

// Rest of the build.gradle.kts

And when I build the project I get:

* Exception is:
Error resolving plugin [id: 'com.test.gradle.myplugin.custom.plugin', version: '0.0.1']
> Could not resolve all dependencies for configuration 'detachedConfiguration1'.
   > Not a supported repository protocol 'artifactregistry': valid protocols are [file, http, https, gcs, s3, sftp]

There is also a similar question asked on Stackoverflow. Note that I am using two different repositories. One for my depenecies maven(url = "artifactregistry://us-central1-maven.pkg.dev/my-registry/my-repository-for-dependencies") and the other one for my plugins maven(url = "artifactregistry://us-central1-maven.pkg.dev/my-registry/my-repository-for-gradle-plugins"). I also tried the alternative way of adding the plugin but I could not solve the problem.

yihanzhen commented 2 years ago

Hi @raminqaf thanks for reporting! I'm not immediately sure what's going on here. Let me play around with it and get back to you.

faculbsz-sg commented 2 years ago

Hi @yihanzhen have you had any chance to take a look at this? I'm facing the same issue.

@raminqaf have you found a workaround?

Thanks!

yihanzhen commented 2 years ago

@raminqaf @faculbsz-sg

Can you try 2.1.5? #67 might have fixed it.

nhalase commented 2 years ago

@yihanzhen I'm also seeing this with 2.1.5 on Gradle 7.4.1 and the Kotlin DSL. Here is my settings.gradle.kts file:

pluginManagement {
    repositories {
        maven {
            setUrl("artifactregistry://us-east1-maven.pkg.dev/my-project/my-repo")
        }
        gradlePluginPortal()
    }
}

buildscript {
    repositories {
        gradlePluginPortal()
    }
    dependencies {
        classpath("gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.1.5")
    }
}

apply(plugin = "com.google.cloud.artifactregistry.gradle-plugin")

dependencyResolutionManagement {
    repositories {
        mavenCentral()
        maven {
            setUrl("artifactregistry://us-east1-maven.pkg.dev/my-project/my-repo")
        }
    }
}

rootProject.name = "example"

Output:

Could not resolve all dependencies for configuration ':compileClasspath'.
Not a supported repository protocol 'artifactregistry': valid protocols are [http, https, file, gcs, s3, sftp]

Update: I just tried it in a custom init.gradle.kts file. It gets further, but ultimately the build will fail with the same error message, just later in the Gradle lifecycle.

yihanzhen commented 2 years ago

Can y'all try the following?

Create a init.gradle in the root directory of your project:

initscript {
    repositories {
        maven {
          url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
      classpath "gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.1.5"
    }

}

apply plugin: com.google.cloud.artifactregistry.gradle.plugin.ArtifactRegistryGradlePlugin

And then run ./gradlew --init-script init.gradle build?

I think this will tell gradle to apply the plugin to settings.gradle too and it indeed got me over Not a supported repository protocol 'artifactregistry'.

Edit: the command should be ./gradlew --init-script init.gradle build

faculbsz-sg commented 2 years ago

Still happening, sorry i dont think i could provide a repo to try this out, but basically im having declared the repo in my settings.gradle like the rest of the guys

dependencyResolutionManagement { repositories { mavenCentral() maven { setUrl("artifactregistry://us-east1-maven.pkg.dev/my-project/my-repo") } } }

yihanzhen commented 2 years ago

@faculbsz-sg sorry, the command in the last comment should have been ./gradlew --init-script init.gradle build to build your project or ./gradlew --init-script init.gradle test to also run the tests. Can you try this?

raminqaf commented 2 years ago

@yihanzhen We couldn't wait for the bug to get fixed so we moved to another repository and now it is really hard for me to test it. But I am hyped for the fix!

fluxxion82 commented 1 year ago

@yihanzhen I ran into this issue and I was able to try out using the init script and it appears to work. I ended up putting the init script in my USER_HOME/.gradle/ dir since I'm using Android Studio to build the project. Not sure I love the idea of using an init script long term for my project. Is there another way to get the same results?

jool commented 1 year ago

Hi, this still seems to be an issue with version 2.2.0. Any updates would be greatly appreciated as this hinders us from migrating to Google Artifactory currently.

EDIT: I found a workaround solution that might help. In addition to doing what nhalase did, I also applied the plugin within pluginManagement:

pluginManagement {
    repositories {
        maven {
            setUrl("artifactregistry://us-east1-maven.pkg.dev/my-project/my-repo")
        }
        gradlePluginPortal()
    }
    plugins {
        id("com.google.cloud.artifactregistry.gradle-plugin")
    }
}

buildscript {
    repositories {
        gradlePluginPortal()
    }
    dependencies {
        classpath("gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.1.5")
    }
}

apply(plugin = "com.google.cloud.artifactregistry.gradle-plugin")

dependencyResolutionManagement {
    repositories {
        mavenCentral()
        maven {
            setUrl("artifactregistry://us-east1-maven.pkg.dev/my-project/my-repo")
        }
    }
}
jool commented 1 year ago

While my previous workaround works for most cases, it does not work for pulling version catalogs from artifactregistry:

dependencyResolutionManagement {

    repositories {
        maven(url = "artifactregistry://europe-maven.pkg.dev/projectd/repo")
    }

    versionCatalogs {
        create("mylibs") {
            from("com.foo.source:version-catalog:version")
        }
    }
}
anguzo commented 1 year ago

Indeed it is still an issue in 2.2.0. For Android application I defined setting.gradle as @jool suggested:

pluginManagement {
    repositories {
        maven {
            url "artifactregistry://europe-north1-maven.pkg.dev/my-project/my-repo"
        }
        google()
        mavenCentral()
        gradlePluginPortal()
    }
    plugins {
        id "com.google.cloud.artifactregistry.gradle-plugin"
    }
}

buildscript {
    repositories {
        google()
        mavenCentral()
        gradlePluginPortal()
    }
    dependencies {
        classpath "gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.2.0"
    }
}

apply plugin: "com.google.cloud.artifactregistry.gradle-plugin"

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        google()
        mavenCentral()
        jcenter()
        maven {
            url "artifactregistry://europe-north1-maven.pkg.dev/my-project/my-repo"
        }
    }
}

But I still get:

Could not resolve all dependencies for configuration ':app:debugRuntimeClasspath'.
Not a supported repository protocol 'artifactregistry': valid protocols are [file, http, https, gcs, s3, sftp]

As a workaround I:

  1. I defined settings.gradle the following way:
    
    pluginManagement {
    repositories {
        google()
        mavenCentral()
        gradlePluginPortal()
    }
    }
    dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.PREFER_PROJECT)
    repositories {
        google()
        mavenCentral()
        jcenter()
        maven {
            url "artifactregistry://europe-north1-maven.pkg.dev/my-project/my-repo"
        }
    }
    }

include ":app" rootProject.name = "App Name"

2. set ```RepositoriesMode``` to ```PREFER_PROJECT``` 
3. registered plugin in top-level ```build.gradle``` ```id 'com.google.cloud.artifactregistry.gradle-plugin' version '2.2.0' apply false```
4. registered plugin in :app ```build.gradle``` ```id 'com.google.cloud.artifactregistry.gradle-plugin'```
5. defined repos in :app ```build.gradle```

repositories { google() mavenCentral() jcenter() maven { url 'artifactregistry://europe-north1-maven.pkg.dev/my-project/my-repo' } }



When this issue is resolved I will get rid of duplicate repos definition in :app ```build.gradle```, but for now that works.
cpaleop commented 11 months ago

Any updates on this? We are still facing this issue on version 2.2.1 and using the init script is not so elegant I might say

cpaleop commented 10 months ago

One more thing; does it make sense that the solution with init.gradle doesn't work when I'm using plugins in my settings.gradle.kts?

pluginManagement {
    repositories {
        google()
        mavenCentral()
        gradlePluginPortal()
        maven {
            setUrl("artifactregistry://europe-maven.pkg.dev/my-project/my-repo")
        }
    }
}

plugins {
    id("com.android.settings") version "8.1.1"
}

Using the init.gradle @yihanzhen provided with the above setup the com.android.settings plugin cannot be resolved, error is:

* What went wrong:
Error resolving plugin [id: 'com.android.settings', version: '8.1.1']
> Could not resolve all dependencies for configuration 'detachedConfiguration1'.
   > Not a supported repository protocol 'artifactregistry': valid protocols are [file, http, https, gcs, s3, sftp]

But when I remove the com.android.settings plugin it works as expected.

cpaleop commented 10 months ago

Just posting an update regarding my previous comment since I did some digging. The way this plugin is implemented will never work if you have plugins declared in your settings.gradle(.kts).

Gradle is looking-up the com.android.settings plugin in the repositories before this plugin has done it's work (which is to change the artifactregistry scheme to https and authenticated it).

This makes sense since this plugin is doing its work after the settings.gradle script has been evaluated once, see relevant code.

I tried to patch the plugin and use beforeSettings instead of settingsEvaluted, but "it's too early" during the evaluation phase of the settings.gradle.kts so I get an empty list of repositories declared.

I don't see how this plugin can work for this case, since Gradle does not provide an API to alter pluginManagement {} before plugins {} block is evaluated.

raminqaf commented 10 months ago

Just posting an update regarding my previous comment since I did some digging. The way this plugin is implemented will never work if you have plugins declared in your settings.gradle(.kts).

Gradle is looking-up the com.android.settings plugin in the repositories before this plugin has done it's work (which is to change the artifactregistry scheme to https and authenticated it).

This makes sense since this plugin is doing its work after the settings.gradle script has been evaluated once, see relevant code.

I tried to patch the plugin and use beforeSettings instead of settingsEvaluted, but "it's too early" during the evaluation phase of the settings.gradle.kts so I get an empty list of repositories declared.

I don't see how this plugin can work for this case, since Gradle does not provide an API to alter pluginManagement {} before plugins {} block is evaluated.

@cpaleop Thanks for the explanation. So, if you want to use your com.android.setting, you'll have to define the plugin in build.gradle.kts?

thiago-scherrer commented 9 months ago

@anguzo I tried your solution but I still have the same error :/

* What went wrong:
Execution failed for task ':app42:parseReleaseLocalResources'.
> Could not resolve all dependencies for configuration ':app42:androidApis'.
   > Not a supported repository protocol 'artifactregistry': valid protocols are [http, https, file, gcs, s3, sftp]
thiago-scherrer commented 9 months ago

I was able to upload using gradlew like this:

  1. Check if the ENV exists on your machine: echo $GOOGLE_APPLICATION_CREDENTIALS

  2. Add to the build.gradle file at the root of your project:

buildscript {
     repositories {
         maven {
             url "https://plugins.gradle.org/m2/"
         }
         Google()
         mavenCentral()
     }

     dependencies {
         classpath "gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.2.0"
     }
}
  1. Add to your APP build.gradle:
id 'com.google.cloud.artifactregistry.gradle-plugin'
apply from: "$project.rootDir/buildSystem/publish/publish_gcp.gradle"

afterEvaluate{
     publishing {
         publications {
             release(MavenPublication) {
                 groupId = 'foo.bar.app.name'
                 artifactId = 'appName'
                 version = '1.2.3'
                 artifact("$buildDir/YOUR-BUILD-FILE-HERE.aar")
             }

         }
     }
}
  1. Create the buildSystem/publish/publish_gcp.gradle file with:
apply plugin: 'com.google.cloud.artifactregistry.gradle-plugin'

publishing {
     repositories {
         maven {
             url "artifactregistry://YOUR-AR-URL-HERE"
         }
     }
}
gavenkoa commented 9 months ago

@thiago-scherrer afterEvaluate is code smell for Gradle... (( There is no guarantee about the order of evaluation and they officially warn against usage...

thiago-scherrer commented 9 months ago

@thiago-scherrer afterEvaluate is code smell for Gradle... (( There is no guarantee about the order of evaluation and they officially warn against usage...

@gavenkoa

Hello! I didn't know, can you send me the link to the documentation where they don't recommend using it? Thanks

gavenkoa commented 9 months ago

@thiago-scherrer Seems Gradle team is not against afterEvaluate() officially, they just warning about it in 7.x when you use new task avoidance API - they are incompatible:

I picked that idea from official Gradle Discourse, you'll see it in every discussion involving afterEvaluate, like: https://discuss.gradle.org/t/is-project-afterevaluate-the-proper-way-for-gradle-plugin-to-dynamically-create-default-tasks/31349

There are some dedicated blog post about it: https://blog.mbonnin.net/my-life-after-afterevaluate

roar-skinderviken commented 3 months ago

After struggling with discovery of a custom plugin in artifactregistry, I ended up with this solution in settings.gradle.kts. Please note useModule.


pluginManagement {
    resolutionStrategy {
        eachPlugin {
            if (requested.id.namespace == "no.mycompany.myplugin") {
                useModule("no.mycompany.mygroup:my-maven-artifact")
            }
        }
    }
    repositories {
        gradlePluginPortal()
        mavenLocal()
        maven {
            url = uri("artifactregistry://europe-north1-maven.pkg.dev/artifact-registry-xxxxx/maven-releases")
        }
    }
}
buildscript {
    repositories {
        gradlePluginPortal()
    }
    dependencies {
        classpath("gradle.plugin.com.google.cloud.artifactregistry:artifactregistry-gradle-plugin:2.2.1")
    }
}
apply(plugin = "com.google.cloud.artifactregistry.gradle-plugin")
robbadler commented 1 month ago

My application also has custom plugins that are loaded in our settings.gradle as

pluginManagement {
  includeBuild('build-tools/plugin-1')
  includeBuild('build-tools/plugin-2')
}

We are not trying to get plugins from artifactregistry, only regular implementation dependencies, but our build does give the same error:

   > Not a supported repository protocol 'artifactregistry': valid protocols are [http, https, file, gcs, s3, sftp]