[Bug] Android Gradle project repositories block don't work for the latest Unity versions

256p commented 1 year ago

[REQUIRED] Please fill in the following fields:

Unity editor version: 2022.2.2f1
External Dependency Manager version: 1.2.175
Source you installed EDM4U: .unitypackage
Features in External Dependency Manager in use: Android Resolver
Plugins SDK in use: None
Platform you are using the Unity editor on: Mac

[REQUIRED] Please describe the issue here:

(Please list the full steps to reproduce the issue. Include device logs, Unity logs, and stack traces if available.)

Steps to reproduce:

Add Dependencies.xml which has custom <repository> specified.
Enable custom main gradle template and custom grade properties template.

Run Android resolver through Assets > External Dependency Manager > Android Resolver > Resolve or Force Resolve.

Expected:

Successful build

Observed:

A lot of warnings from Gradle like that:

Build was configured to prefer settings repositories over project repositories but repository 'maven' was added by build file 'unityLibrary/build.gradle'

And this error:

Execution failed for task ':launcher:checkReleaseDuplicateClasses'.
> Could not resolve all dependencies for configuration ':launcher:releaseRuntimeClasspath'.
The project declares repositories, effectively ignoring the repositories you have declared in the settings.
You can figure out how project repositories are declared by configuring your build to fail on project repositories.
See https://docs.gradle.org/7.2/userguide/declaring_repositories.html#sub:fail_build_on_project_repositories for details.
> Could not find com.ironsource.sdk:mediationsdk:7.2.7.
 Searched in the following locations:
   - https://dl.google.com/dl/android/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
   - https://repo.maven.apache.org/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
   - file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk-7.2.7.jar
   - file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk.jar
 Required by:
     project :launcher > project :unityLibrary

You can use this project for testing: https://github.com/256p/unity-jar-resolver-custom-template.git

The reason behind this is that some time ago Unity switched to the new Android Gradle Plugin version (7.1.0). In this version repository setting was moved from the top-level build.gradle file to the setting.gradle (release notes: https://developer.android.com/studio/past-releases/past-agp-releases/agp-7-1-0-release-notes#settings-gradle). But EDM4U still adds project.repositories block in the main gradle template which will not work if settings.gradle has dependencyResolutionManagement block.

As I see there are several possible solutions:

Use IPostGenerateGradleAndroidProject.OnPostGenerateGradleAndroidProject and modify settings.gradle after every build.
Generate settingsTemplate.gradle automatically and add repositories there instead of the main gradle template.
Ask the user to create settingsTemplate.gradle and then generate a repositories list there. But Unity doesn't provide a checkbox for the settings template (which is quite odd), which means that the user will need to copy the correct template from PlaybackEngines/AndroidPlayer/Tools/GradleTemplates/.
Or maybe some other solution.

Please answer the following, if applicable:

What's the issue repro rate? 100%

What happened? How can we make the problem occur? This could be a description, log/console output, etc.

If you have a downloadable sample project that reproduces the bug you're reporting, you will likely receive a faster response on your issue.

google-oss-bot commented 1 year ago

This issue does not seem to follow the issue template. Make sure you provide all the required information.

paulinon commented 1 year ago

Hi @256p,

Thanks for reporting this. Could you provide the Dependencies.xml you used in reproducing the issue? I haven't been successful in replicating the behavior so far. Additionally, I should mention that I've used Unity 2022.2.3f1. Could you confirm if this version of the editor makes any difference?

256p commented 1 year ago

Hi @paulinon,

Here is xml:

<?xml version="1.0" encoding="utf-8"?>
<dependencies>
    <androidPackages>
        <androidPackage spec="com.ironsource.sdk:mediationsdk:7.2.7">
            <repositories>
                <repository>https://android-sdk.is.com/</repository>
            </repositories>
        </androidPackage>
    </androidPackages>
</dependencies>

I will try 2022.2.3f1

256p commented 1 year ago

Hi @paulinon,

I have managed to reproduce it with 2022.2.3f1 Here exact steps that I performed:

Cloned https://github.com/256p/unity-jar-resolver-custom-template.git
Opened project with Unity 2022.2.3f1 for Android
Launched Build & Run
EDM4U should resolve on the build but I also tried Force Resolve and built a second time and got the same error Here is full error logs:
```
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
```

FAILURE: Build failed with an exception.

What went wrong: Execution failed for task ':launcher:checkReleaseDuplicateClasses'.
Could not resolve all dependencies for configuration ':launcher:releaseRuntimeClasspath'. The project declares repositories, effectively ignoring the repositories you have declared in the settings. You can figure out how project repositories are declared by configuring your build to fail on project repositories. See https://docs.gradle.org/7.2/userguide/declaring_repositories.html#sub:fail_build_on_project_repositories for details. Could not find com.ironsource.sdk:mediationsdk:7.2.7. Searched in the following locations:
- https://dl.google.com/dl/android/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
- https://repo.maven.apache.org/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
- file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk-7.2.7.jar
- file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk.jar Required by: project :launcher > project :unityLibrary
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 1s

UnityEditor.EditorApplication:Internal_CallGlobalEventHandler () (at /Users/bokken/build/output/unity/unity/Editor/Mono/EditorApplication.cs:473)

256p commented 1 year ago

Sorry, I didn't copy the whole log. Here is full log:

CommandInvokationFailure: Gradle build failed. 
/Applications/Unity/Hub/Editor/2022.2.3f1/PlaybackEngines/AndroidPlayer/OpenJDK/bin/java -classpath "/Applications/Unity/Hub/Editor/2022.2.3f1/PlaybackEngines/AndroidPlayer/Tools/gradle/lib/gradle-launcher-7.2.jar" org.gradle.launcher.GradleMain "-Dorg.gradle.jvmargs=-Xmx4096m" "assembleRelease"

Environment Variables:
BURST_ANDROID_MIN_API_LEVEL = 22
MallocSpaceEfficient = 1
EXTENSION_KIT_EXTENSION_TYPE = 2
XPC_FLAGS = 0x0
JAVA_TOOL_OPTIONS = -Dfile.encoding=UTF-8
MallocNanoZone = 0
ORIGINAL_XDG_CURRENT_DESKTOP = undefined
SHELL = /bin/zsh
SSH_AUTH_SOCK = /private/tmp/com.apple.launchd.29IzpVmxG8/Listeners
HOME = /Users/stanislavspetkevics
PATH = /usr/bin:/bin:/usr/sbin:/sbin
__CFBundleIdentifier = com.unity3d.unityhub
USER = stanislavspetkevics
__CF_USER_TEXT_ENCODING = 0x1F5:0x0:0x2
TMPDIR = /var/folders/c6/8h8wwy1x3sg7jpgr8ymrq8xm0000gn/T/
LOGNAME = stanislavspetkevics
JAVA_HOME = /Applications/Unity/Hub/Editor/2022.2.3f1/PlaybackEngines/AndroidPlayer/OpenJDK
ANDROID_NDK_ROOT = /Applications/Unity/Hub/Editor/2022.2.3f1/PlaybackEngines/AndroidPlayer/NDK
XPC_SERVICE_NAME = application.com.unity3d.unityhub.67105553.67105559
COMMAND_MODE = unix2003

stderr[
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':launcher:checkReleaseDuplicateClasses'.
> Could not resolve all dependencies for configuration ':launcher:releaseRuntimeClasspath'.
  The project declares repositories, effectively ignoring the repositories you have declared in the settings.
  You can figure out how project repositories are declared by configuring your build to fail on project repositories.
  See https://docs.gradle.org/7.2/userguide/declaring_repositories.html#sub:fail_build_on_project_repositories for details.
   > Could not find com.ironsource.sdk:mediationsdk:7.2.7.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
       - https://repo.maven.apache.org/maven2/com/ironsource/sdk/mediationsdk/7.2.7/mediationsdk-7.2.7.pom
       - file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk-7.2.7.jar
       - file:/Users/stanislavspetkevics/Desktop/unity-jar-resolver-custom-template/Library/Bee/Android/Prj/Mono2x/Gradle/unityLibrary/libs/mediationsdk.jar
     Required by:
         project :launcher > project :unityLibrary

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':unityLibrary:generateReleaseRFile'.
> Could not resolve all dependencies for configuration ':unityLibrary:releaseCompileClasspath'.
  The project declares repositories, effectively ignoring the repositories you have declared in the settings.
  You can figure out how project repositories are declared by configuring your build to fail on project repositories.
  See https://docs.gradle.org/7.2/userguide/declaring_repositories.html#sub:fail_build_on_project_repositories for details.
   > Could not find com.ironsource.sdk:mediationsdk:7.2.7.
     Required by:
         project :unityLibrary

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
]
stdout[

> Configure project :unityLibrary
Build was configured to prefer settings repositories over project repositories but repository 'maven' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'maven2' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenLocal' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenRepo' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'maven' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'maven2' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenLocal' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenRepo' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'maven' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'maven2' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenLocal' was added by build file 'unityLibrary/build.gradle'
Build was configured to prefer settings repositories over project repositories but repository 'MavenRepo' was added by build file 'unityLibrary/build.gradle'

> Task :launcher:preBuild UP-TO-DATE
> Task :unityLibrary:preBuild UP-TO-DATE
> Task :launcher:preReleaseBuild UP-TO-DATE
> Task :unityLibrary:preReleaseBuild UP-TO-DATE
> Task :unityLibrary:compileReleaseAidl NO-SOURCE
> Task :unityLibrary:mergeReleaseJniLibFolders UP-TO-DATE
> Task :launcher:generateReleaseBuildConfig UP-TO-DATE
> Task :launcher:compileReleaseAidl NO-SOURCE
> Task :unityLibrary:mergeReleaseNativeLibs UP-TO-DATE
> Task :launcher:javaPreCompileRelease UP-TO-DATE
> Task :launcher:writeReleaseApplicationId UP-TO-DATE
> Task :unityLibrary:stripReleaseDebugSymbols UP-TO-DATE
> Task :launcher:analyticsRecordingRelease
> Task :launcher:generateReleaseResValues UP-TO-DATE
> Task :unityLibrary:copyReleaseJniLibsProjectAndLocalJars UP-TO-DATE
> Task :unityLibrary:compileReleaseRenderscript NO-SOURCE
> Task :launcher:createReleaseCompatibleScreenManifests UP-TO-DATE
> Task :launcher:extractDeepLinksRelease UP-TO-DATE
> Task :unityLibrary:generateReleaseBuildConfig UP-TO-DATE
> Task :unityLibrary:generateReleaseResValues UP-TO-DATE
> Task :launcher:extractProguardFiles UP-TO-DATE
> Task :unityLibrary:generateReleaseResources UP-TO-DATE
> Task :launcher:mergeReleaseJniLibFolders UP-TO-DATE
> Task :unityLibrary:packageReleaseResources UP-TO-DATE
> Task :unityLibrary:parseReleaseLocalResources UP-TO-DATE
> Task :unityLibrary:processReleaseManifest UP-TO-DATE
> Task :launcher:checkReleaseDuplicateClasses FAILED
> Task :unityLibrary:generateReleaseRFile FAILED
20 actionable tasks: 3 executed, 17 up-to-date
]
exit code: 1
UnityEditor.Android.Command.WaitForProgramToRun (UnityEditor.Utils.Program p, UnityEditor.Android.Command+WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.Command.Run (System.Diagnostics.ProcessStartInfo psi, UnityEditor.Android.Command+WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.Command.Run (System.String command, System.String args, System.String workingdir, UnityEditor.Android.Command+WaitingForProcessToExit waitingForProcessToExit, System.String errorMsg) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.AndroidJavaTools.RunJava (System.String args, System.String workingdir, System.Action`1[T] progress, System.String error) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.GradleWrapper.Run (UnityEditor.Android.AndroidJavaTools javaTools, Unity.Android.Gradle.AndroidGradle androidGradle, System.String workingdir, System.String task, System.Action`1[T] progress) (at <1fd6adee695040e688c666557c8c18d0>:0)
Rethrow as GradleInvokationException: Gradle build failed
UnityEditor.Android.GradleWrapper.Run (UnityEditor.Android.AndroidJavaTools javaTools, Unity.Android.Gradle.AndroidGradle androidGradle, System.String workingdir, System.String task, System.Action`1[T] progress) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.PostProcessor.Tasks.BuildGradleProject.Execute (UnityEditor.Android.PostProcessor.PostProcessorContext context) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.Android.PostProcessor.PostProcessRunner.RunAllTasks (UnityEditor.Android.PostProcessor.PostProcessorContext context) (at <1fd6adee695040e688c666557c8c18d0>:0)
UnityEditor.EditorApplication:Internal_CallGlobalEventHandler() (at /Users/bokken/build/output/unity/unity/Editor/Mono/EditorApplication.cs:473)

256p commented 1 year ago

Hi @paulinon,

I just noticed that I haven't mentioned in the issue description the last step. You need to build a project for Android. Resolving isn't failing, the problem is the outdated Gradle script that EDM4U writes to the main template that crashes game build.

paulinon commented 1 year ago

Thanks for the additional input, @256p. I was able to reproduce the issue you're facing. Let me bring this up to the team. That said, I'll be marking this as a bug for now.

kjt0429 commented 1 year ago

i have a same issue.

jonsimantov commented 1 year ago

As a workaround while we look into this, you can try having Unity use an older Gradle plugin version by going to Preferences / External Tools and unchecking "Gradle installed with Unity (recommended)", then click Browse to locate an older Gradle plugin on your machine. Especially if you have an older Unity version installed, you can navigate to that version's Gradle plugin.

256p commented 1 year ago

Also, there is another workaround that keeps the Gradle version (in case some android lib requires it). For that workaround, you will need to copy a file from /Applications/Unity/Hub/Editor/2022.2.3f1/PlaybackEngines/AndroidPlayer/Tools/GradleTemplates/settingsTemplate.gradle to Assets/Plugins/Android/settingsTemplate.gradle. That way Unity will use your template in Assets/Plugins/Android. (It's weird that Unity doesn't has a checkbox for that) For my version of unity, it looks like that:

pluginManagement {
    repositories {
        **ARTIFACTORYREPOSITORY**
        gradlePluginPortal()
        google()
        mavenCentral()
    }
}

include ':launcher', ':unityLibrary'
**INCLUDES**

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.PREFER_SETTINGS)
    repositories {
        **ARTIFACTORYREPOSITORY**
        google()
        mavenCentral()
        flatDir {
            dirs "${project(':unityLibrary').projectDir}/libs"
        }
    }
}

The conflicting part is RepositoriesMode.PREFER_SETTINGS. But you can safely delete the whole dependencyResolutionManagement block from here. Since the android project still needs to know about local dependencies, copy the repositories block, you will need it later. Now settingsTemplate.gradle should look like this:

pluginManagement {
    repositories {
        **ARTIFACTORYREPOSITORY**
        gradlePluginPortal()
        google()
        mavenCentral()
    }
}

include ':launcher', ':unityLibrary'
**INCLUDES**

Now in Project Settings > Player > Publishing Settings for Android enable Custom Base Gradle Template Initially, it will look like that:

plugins {
    id 'com.android.application' version '7.1.2' apply false
    id 'com.android.library' version '7.1.2' apply false
    **BUILD_SCRIPT_DEPS**
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

Now create allprojects block and paste repositories copied in the settings template. As a result base template will look like that:

plugins {
    id 'com.android.application' version '7.1.2' apply false
    id 'com.android.library' version '7.1.2' apply false
    **BUILD_SCRIPT_DEPS**
}

allprojects {
    repositories {
        **ARTIFACTORYREPOSITORY**
        google()
        mavenCentral()
        flatDir {
            dirs "${project(':unityLibrary').projectDir}/libs"
        }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

Now Gradle should build successfully.

IvanArantes commented 1 year ago

Followed 256p steps and it worked 100% thanks!

dreamcodestudio commented 1 year ago

inserting custom settingsTemplate.gradle by @256p on Editor 2022.2.8f1 works fine too

valonsoft commented 1 year ago

Thanks @256p Your solution worked perfectly!

256p commented 1 year ago

Hi, @paulinon you can check PR with possible solution https://github.com/googlesamples/unity-jar-resolver/pull/605

berkceb commented 1 year ago

I have came across a similar error with different third party repository. Using @256p 's solution worked in Unity 2022.2.13.

googlesamples / unity-jar-resolver