search

dotnet / android

.NET for Android provides open-source bindings of the Android SDK for use with .NET managed languages such as C#
MIT License
1.92k stars 526 forks source link

`InstallAndroidDependencies` target doesn't install platform-tools and other necessary parts of the SDK. #7284

Closed Lauriethefish closed 1 week ago

Lauriethefish commented 2 years ago

Android application type

Android for .NET (net6.0-android, etc.)

Affected platform version

.NET 6.0.400

Description

When installing the android SDK using the InstallAndroidDependencies target, the downloaded SDK is missing key tools, such as aapt and platform-tools:

laurie@d95389ff63a7:/opt/android-sdk$ ls
platforms

This causes a build with the installed SDK to fail with error XA5300, since Xamarin discards your selected SDK directory when it thinks it's invalid. (perhaps a better error message would be great here, but not super important for this issue).

It would be great if this could be fixed since this target is very, very helpful for CI.

Steps to Reproduce

  1. Start a docker container with the mcr.microsoft.com/dotnet/sdk:6.0 image.
  2. Create a new user, make them a sudoer, and log into the user with su.
  3. Install the android workload sudo dotnet workload install android.
  4. Create a new net6.0-android project: sudo dotnet new android -o example && cd example.
  5. Restore the project: sudo dotnet restore ./example.csproj
  6. Invoke InstallAndroidDependencies to download the SDK: sudo dotnet msbuild ./example.csproj -t:InstallAndroidDependencies -p:AndroidSdkDirectory=/opt/android-sdk -p:AcceptAndroidSDKLicenses=true
  7. Attempt to use the new android SDK directory to build the project: sudo dotnet publish -c Release -p:AndroidSdkDirectory=/opt/android-sdk. You will then receive the error: /usr/share/dotnet/packs/Microsoft.Android.Sdk.Linux/32.0.448/tools/Xamarin.Android.Tooling.targets(69,5): error XA5300: The Android SDK directory could not be found. Check that the Android SDK Manager in Visual Studio shows a valid installation. To use a custom SDK path for a command line build, set the 'AndroidSdkDirectory' MSBuild property to the custom path. [/home/laurie/example/example.csproj]

Did you find any workaround?

No workaround currently, I tried installing the SDK with sudo apt-get install -y android-sdk, but unfortunately that SDK only targets API level 29, and we need API level 31 for net6.0-android.

Relevant log output

See comment for diagnostic logs of both InstallAndroidDepencies and publish.

Lauriethefish commented 2 years ago

Logs here. InstallAndroidDependencies.log Publish.log

Lauriethefish commented 1 year ago

Hello, it's been a while, any chance this could be triaged?

jonathanpeppers commented 1 year ago

@dellis1972 there are some interesting logs like:

Could not retrieve Pkg.Revision from /opt/android-sdk/tools/source.properties, r0 assumed (TaskId:21)

Is it possible you can't build Android projects without tools installed? I'm not sure if that is the underlying issue, or if it just couldn't install tools.

dellis1972 commented 1 year ago

I have no idea. From the log it looks like we are calculating the dependencies correctly. 

Dependencies=
   platforms/android-31
   build-tools/32.0.0
           Version=32.0.0
   platform-tools
           Version=33.0.2
   cmdline-tools/7.0
           Version=7.0

We will probably need to get someone to look at the SDKManager code to see what is happening.

jonpryor commented 1 year ago

@Lauriethefish: is /opt/android-sdk writable by your user?

Some of the output of InstallAndroidDependencies.log looks reasonable:

Component Android SDK Platform-Tools r31.0.3 not present on the system

in that we detected that Platform-Tools wasn't present. Yay.

What's odd is that the Dependencies= line that @dellis1972 noted, which lists platform-tools Version=33.0.2. 33.0.2 is not 31.0.3, so…why are we getting a log message mentioning Platform-Tools r31.0.3 and not 33.0.2?

That might point to the problem, though: .NET 6 InstallAndroidDependencies is using the "xamarin manifest" at https://aka.ms/xamarin/sdkmanifest/d17-1, which only has platform-tools 31.0.3:

    <platform-tools revision="31.0.3" path="platform-tools" filesystem-path="platform-tools" manifest-url="https://dl.google.com/android/repository/repository2-1.xml" description="Android SDK Platform-Tools" obsolete="False" preview="False" license="android-sdk-license" original-type="generic:genericDetailsType">
        <urls>
            <url host-os="macosx" host-bits="0" size="13227985" checksum-type="sha1" checksum="15f6f7e97b35994d538a0fc5147ad5fb502ba03d">https://dl.google.com/android/repository/e8b2b4cbe47c728c1e54c5f524440b52d4e1a33c.platform-tools_r31.0.3-darwin.zip</url>
            <url host-os="linux" host-bits="0" size="13302579" checksum-type="sha1" checksum="f09581347ed39978abb3a99c6bb286de6adc98ef">https://dl.google.com/android/repository/platform-tools_r31.0.3-linux.zip</url>
            <url host-os="windows" host-bits="0" size="11912013" checksum-type="sha1" checksum="26bc02bbd920e8ed461ae526cc4c69d773b72395">https://dl.google.com/android/repository/platform-tools_r31.0.3-windows.zip</url>
        </urls>
    </platform-tools>

The <CalculateProjectDependencies/> task, meanwhile, says that platform-tools 33.0.2 is a dependency:

Task "CalculateProjectDependencies" (TaskId:20)
  Task Parameter:TargetFrameworkVersion=v6.0 (TaskId:20)
  Task Parameter:BuildToolsVersion=32.0.0 (TaskId:20)
  Task Parameter:ManifestFile=/home/laurie/example/AndroidManifest.xml (TaskId:20)
  Task Parameter:AndroidApiLevel=31 (TaskId:20)
  Task Parameter:PlatformToolsVersion=33.0.2 (TaskId:20)
  Task Parameter:NdkRequired=False (TaskId:20)
  Task Parameter:NdkVersion=24.0.8215888 (TaskId:20)
  Task Parameter:CommandLineToolsVersion=7.0 (TaskId:20)
  Output Item(s): 
      AndroidDependency=
          platforms/android-31
          build-tools/32.0.0
                  Version=32.0.0
          platform-tools
                  Version=33.0.2
          cmdline-tools/7.0
                  Version=7.0 (TaskId:20)
Done executing task "CalculateProjectDependencies". (TaskId:20)

Platform-tools 33.0.2, meanwhile, comes from PlatformToolsVersion=33.0.2, which is from $(AndroidSdkPlatformToolsVersion):

https://github.com/xamarin/xamarin-android/blob/release/6.0.4xx/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets#L2633C29-L2633C59

$(AndroidSdkPlatformToolsVersion) is from xamarin-android-tools:

https://github.com/xamarin/xamarin-android-tools/blob/9c641b3e08e56db37467a64a2c5de2c7f7ddb3ef/src/Xamarin.Android.Tools.AndroidSdk/Xamarin.Android.Tools.Versions.props#L14

So what we have here is an inconsistency between $(AndroidSdkPlatformToolsVersion) and the xamarin-manifest.

A potential workaround for @Lauriethefish would be to override things so that platform-tools 31.0.3 is used, e.g. editing the .csproj to have:

<PropertyGroup>
  <AndroidSdkPlatformToolsVersion>31.0.3</AndroidSdkPlatformToolsVersion>
</PropertyGroup>

What we need to do is:

  1. Do a better job of ensuring consistency: $(AndroidSdkPlatformToolsVersion) must be a value within the xamarin-manifest.

    For awhile, we were only building with the GoogleV2 manifest, not the Xamarin manifest, but we now build against both, so presumably CI will fail things are inconsistent:

    https://github.com/xamarin/xamarin-android/blob/6768c731d327c8148c45304c895ca8987a9cc2f1/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidDependenciesTests.cs#L20-L82

  2. The messages produced by <InstallAndroidDependencies/> aren't great. Why is "Platform-Tools r31.0.3" mentioned at all? (Because it's checking what's installed against what's in the manifest.) Which manifest is being used? (How does one know that .NET 6 is using the manifest at https://aka.ms/xamarin/sdkmanifest/d17-1 without being us?) Why doesn't it verify that what was requested is actually installed, and fail if it isn't?

Lauriethefish commented 1 year ago

@jonpryor

is /opt/android-sdk writable by your user?

It is. Looking through the dockerfile I was debugging, msbuild is invoked as root.

jonpryor commented 1 year ago

@Lauriethefish: did you try the potential workaround I mentioned, updating your .csproj to contain:

<PropertyGroup>
  <AndroidSdkPlatformToolsVersion>31.0.3</AndroidSdkPlatformToolsVersion>
</PropertyGroup>
Lauriethefish commented 1 year ago

I will try it in a few hours and get back to you.

Lauriethefish commented 1 year ago

@jonpryor The workaround did seem to fix the error of the SDK not being recognised, however, some parts of the SDK are still missing, e.g. aapt, which causes a new error. InstallAndroidDependencies.log publish.log

jonathanpeppers commented 1 year ago

Are you using AndroidUseAapt2=false anywhere? The log isn't diagnositc, so I am not able to tell. I would remove this setting if you have it.

Do things fare any better if you use .NET 7 instead? The .NET 6 android workload is out of support:

Microsoft.NET.EolTargetFrameworks.targets(35,5): warning NETSDK1202: The workload 'android' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/maui-support-policy for more information about the support policy.
Lauriethefish commented 1 year ago

I'm not using AndroidUseAapt2=false anywhere, no. Strange, I thought I had set the log to diagnostic. Yeah, I noticed the workload was out of support. I will give .NET 7 a go at some point, out of time today though.

jonathanpeppers commented 1 year ago

It does seem like we should remove the logic looking for aapt here:

https://github.com/xamarin/xamarin-android/blob/a48a9c564c3ad5e18f1076c7521379fbfe6d669d/src/Xamarin.Android.Build.Tasks/Tasks/ResolveAndroidTooling.cs#L136-L140

And look for aapt2 instead, but I do see it on disk:

image

dellis1972 commented 1 year ago

Looking at the log, it seems that the build-tools are not being installed.

Android package has no revision defined
         Could not retrieve Pkg.Revision from /opt/android-sdk/tools/source.properties, r0 assumed
         Dependency found: Android SDK Platform 31
         Dependency found: Android SDK Platform-Tools
         Adding component 'Android SDK Platform 31 r1 [Platform: API 31]'
         Adding component 'Android SDK Platform-Tools r31.0.3'
         Dependency to be installed: Android SDK Platform 31
         Dependency to be installed: Android SDK Platform-Tools

Further up we have

         Component Android SDK Platform-Tools r31.0.3 not present on the system
         Component Android SDK Command-line Tools r5.0 not present on the system
         Component Android SDK Command-line Tools (latest) r5.0 not present on the system
         Component Android SDK Build-Tools 30.0.3 r30.0.3 not present on the system
         Component Android SDK Build-Tools 30.0.2 r30.0.2 not present on the system
         Component Android Emulator r31.2.9 not present on the system
         Component Android Emulator r31.1.4 not present on the system

its likely you will need to override AndroidSdkBuildToolsVersion property as well to use one which exists for the platform-tools you are trying to instll. You will probably need to define AndroidCommandLineToolsVersion as well.

jonpryor commented 1 week ago

As the investigation was (partially) verified, I'll assert that the issue is that our various MSBuild properties which control component versions was out of sync with the manifest used for installation.

This is something we're still improving (see #9155), but I think that this issue is currently fixed for .NET 8, and our unit tests should help keep things valid.