status-im / status-mobile

a free (libre) open source, mobile OS for Ethereum
https://status.app
Mozilla Public License 2.0
3.85k stars 983 forks source link

No Android SDK available for Nix builds on Darwin aarch64 #12794

Open jakubgs opened 2 years ago

jakubgs commented 2 years ago

When @Jonathansoufer tried to build on his Mac with M1 aarch64 CPU it fails with:

No Android SDK tarballs are available for system architecture: aarch64-darwin

Which is caused by this if in Android SDK derivation:

{
  # Determine the Android os identifier from Nix's system identifier
  os = if stdenv.system == "x86_64-linux" then "linux"
    else if stdenv.system == "x86_64-darwin" then "macosx"
    else throw "No Android SDK tarballs are available for system architecture: ${stdenv.system}";
}

https://github.com/NixOS/nixpkgs/blob/65135081/pkgs/development/mobile/androidenv/compose-android-packages.nix#L31-L34

Since currently packages do not appear to include any aarch64-darwin builds: https://github.com/NixOS/nixpkgs/blob/65135081/pkgs/development/mobile/androidenv/repo.json

EDIT: The current solution is to simply not use an arm64 shell to run builds, but instead use the Rosetta 2 dynamic binary translator through the arch -x86_64 command or Terminal x86 application.

jakubgs commented 2 years ago

There appears to be an Android Studio release build for mac_arm:

image

jakubgs commented 2 years ago

I run the generate.sh script to get available packages, the XML files from Google do not appear to include any ARM builds:

 > grep -iE '(aarch64|arm64)' repository2-1.xml 

 > grep -iE '(aarch64|arm64)' addon2-1.xml     
jakubgs commented 2 years ago

And the mkrepo.rb script does not appear to have any lines referencing specific architectures.

jakubgs commented 2 years ago

But there's more to this than just not being able to build Android on ARM64 MacOS.

The issue is that gomobile necessary to build status-go on MacOS for iOS is currently built by default with support for both iOS and Android platforms, which makes it require Android SDK at build time, which triggers the dependency on Android SDK even if we're just building for iOS.

jakubgs commented 2 years ago

I've made a fix for Gomobile builds that makes Android SDK optional: https://github.com/status-im/status-react/pull/12797

chayleaf commented 2 years ago

Do I understand correctly that NixOS uses binary SDK releases, despite it being open-source?

jakubgs commented 2 years ago

We use what nixpkgs provides, which is using the distribution provided by Google:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/mobile/androidenv/repo.json https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/mobile/androidenv/fetchrepo.sh

Now, it would be possible to build our own, I see two options:

This would most probably involve quite a lot of work, but it should be doable in theory. Truth be told, if Android SDK distribution would be compromised anyone and everyone would be fucked, since I do not know of any project that builds their own Android SDK. Would be interesting to see some examples tho.

chayleaf commented 2 years ago

I was recently bitten by the fact Nix doesn't build Android SDK from source - I wanted to install fdroidserver on an arm64 machine, but it has an optional (but recommended) dependency on apksigner. I simply removed the dependency with an overlay, but it's a suboptimal solution. There probably isn't a project that builds the entire SDK from source (There used to be one before), but some parts are certainly built from source by other distros. See https://github.com/nmeum/android-tools

Also, the binary builds are non-free (it requires you to accept an EULA), but the source code is free software - which is why I asked the question. As far as I can tell, the packages are properly marked as unfree in https://github.com/NixOS/nixpkgs/blob/6b6d2abfa0cd7d8c64a79647ca6bcac64e3695d9/pkgs/development/mobile/androidenv/tools/26.nix#L39, but I'm not sure if all of them are

jakubgs commented 2 years ago

We don't consider implementing Android SDK builds from source as something high priority.

I might look into it eventually, but not anytime soon.

chayleaf commented 2 years ago

that makes sense, i might look into it too at some point, but it doesn't really matter that much for me either

danwdart commented 1 year ago

Still seeing this, which is breaking workflows on this architecture. What can I add to shell.nix to use it properly?

jakubgs commented 1 year ago

If there are no pre-built assets for given OS and architecture than nothing. Aside from trying to build Android SDK for given platform yourself. But good luck with that.

BalogunofAfrica commented 1 year ago

Just another note for anyone who this issue persists for (like me) after using rosetta.

You might have to uninstall nix and reinstall it using the x86 terminal (the rosetta one), then you're good to go.

yakimant commented 1 year ago

Another workaround merged, which doesn't require any setup: https://github.com/status-im/status-mobile/pull/16237

cammellos commented 11 months ago

This seems to have been addressed, closing for now

jakubgs commented 11 months ago

This has not been resolved. We only created a workaround using the NIXPKGS_SYSTEM_OVERRIDE environment variable in:

The actual underlying issue is still there, there is no Android SDK for Darwin aarch64.