search

nix-community / nix-on-droid

Nix-enabled environment for your Android device. [maintainers=@t184256,@Gerschtli]
https://nix-on-droid.unboiled.info
MIT License
1.29k stars 72 forks source link

Any way to deploy from desktop to phone using deploy-rs or nixus? #94

Open bbigras opened 3 years ago

bbigras commented 3 years ago

https://github.com/serokell/deploy-rs https://github.com/Infinisil/nixus

Those tools allow us to build multiples machines at once and push deploy them.

If I can build aarch64, can I just build something, nix-copy it to the phone and run an activate command on the phone?

Gerschtli commented 3 years ago

That should be possible I think. Have a look at nix-on-droid/nix-on-droid.sh on how we build and activate :)

t184256 commented 3 years ago

You'll need an ssh server on the phone if you're keen on initiating that from a desktop, I guess, otherwise it's just https://github.com/t184256/nix-on-droid/wiki/Remote-building.

But once you have it, "build something, nix-copy it to the phone and run an activate command on the phone" should just work, yeah.

bbigras commented 3 years ago

Thanks. I think I'm making some progress.

Any way to force system to aarch64-linux when running on x86_x64?

right now I'm trying to do this:

        profiles.system.path = deploy-rs.lib.aarch64-linux.activate.custom
          (
            (import (nix-on-droid + "/modules") {
              pkgs = nixpkgs.legacyPackages."aarch64-linux";
              config = nix_on_droid_config;
              home-manager-src = home-manager;
            }).activationPackage
          ) "./activate";

I think maybe I have to specify pkgs = nixpkgs.legacyPackages."aarch64-linux" in a couple of places.

current status:

❯ deploy ".#pixel2" -- --impure
🚀 ℹī¸ [deploy] [INFO] Running checks for flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
warning: unknown flake output 'deploy'
🚀 ℹī¸ [deploy] [INFO] Evaluating flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹī¸ [deploy] [INFO] The following profiles are going to be deployed:
[pixel2.system]
user = "nix-on-droid"
ssh_user = "nix-on-droid"
path = "/nix/store/87hfvlqrb2760ns1jqz6x7b0zdfrn5cm-activatable-nix-on-droid-generation"
hostname = "pixel2"
ssh_opts = []

🚀 ℹī¸ [deploy] [INFO] Building profile `system` for node `pixel2`
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹī¸ [deploy] [INFO] Activating profile `system` for node `pixel2`
⭐ ℹī¸ [activate] [INFO] Activating profile
/nix/var/nix/profiles/per-user/nix-on-droid/system/deploy-rs-activate: line 2:  8338 Illegal instruction     ./activate
⭐ ❌ [activate] [ERROR] The activation script resulted in a bad exit code: Some(132)
🚀 ❌ [deploy] [ERROR] Failed to deploy profile: Activating over SSH resulted in a bad exit code: Some(1)

In result, it seems my nix-on-droid-path has x86_64 stuff.

activate-rs is aarch64

EDIT: ok nix-on-droid-path is now aarch64

bbigras commented 3 years ago

EDIT: I'm calling ./activate without the full path.

Current status:

❯ deploy ".#pixel2" -- --impure
🚀 ℹī¸ [deploy] [INFO] Running checks for flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
warning: unknown flake output 'deploy'
🚀 ℹī¸ [deploy] [INFO] Evaluating flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹī¸ [deploy] [INFO] The following profiles are going to be deployed:
[pixel2.system]
user = "nix-on-droid"
ssh_user = "nix-on-droid"
path = "/nix/store/zfyia9axhilazg3l1qw8vhi2n2raqj2d-activatable-nix-on-droid-generation"
hostname = "pixel2"
ssh_opts = []

🚀 ℹī¸ [deploy] [INFO] Building profile `system` for node `pixel2`
warning: Git tree '/home/bbigras/nix-config' is dirty
copying path '/nix/store/2jjf28m0qvn9cfqrii9gzav5msv0nf0r-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'...
copying path '/nix/store/q3acm62dxjdw7rfgnz4zyiz2j619ngr3-glibc-locales-2.32-10' from 'https://cache.nixos.org'...
copying path '/nix/store/qq58n27vwpbhqbzzrzwq0gvl64nl7v74-less-563' from 'https://cache.nixos.org'...
copying path '/nix/store/4klvh9qmav5v841rlrnq9hvhz1dvw7ss-vim-8.2.1522' from 'https://cache.nixos.org'...
🚀 ℹī¸ [deploy] [INFO] Activating profile `system` for node `pixel2`
⭐ ℹī¸ [activate] [INFO] Activating profile
Activating linkBinSh
Activating linkUsrBinEnv
Activating installLogin
Activating installLoginInner
Activating installPackages
replacing old 'nix-on-droid-path'
installing 'nix-on-droid-path'
building '/nix/store/a4axx3d179cm9l21qhnz3fj6ms9ifqvx-user-environment.drv'...
created 528 symlinks in user environment
Activating installProotStatic
Activating setUpEtc
Removing obsolete symlink '/etc/profiles/per-user/nix-on-droid'...
Activating linkProfile
error: --set requires exactly one derivation
⭐ ❌ [activate] [ERROR] The activation script resulted in a bad exit code: Some(1)
🚀 ❌ [deploy] [ERROR] Failed to deploy profile: Activating over SSH resulted in a bad exit code: Some(1)
bbigras commented 3 years ago

It works if I run the full path in my phone!

Screenshot_20210111-154506.png

Edit: well the "no change" might not be normal. Maybe I ran the wrong one.

Openssh doesn't seem to accept my ssh key after the switch for some reason.

bbigras commented 3 years ago

Any ideas why the nix-on-droid user would break?

EDIT: oh for some reason the nix-on-droid's uid in /etc/passwd is 1000 when I deploy with deploy-rs and it's 10294 when I use nix-on-droid switch.

EDIT 2: I was able to work around it by hardcoding the "right uid/gid. You can see my fork for my changes.

-bash-4.4$ whoami
-whoami: cannot find name for user ID 10294
Gerschtli commented 3 years ago

The reason for the wrong uid and gid is, that these values have to be generated on the destination machine. There is a derivation built in modules/user.nix, which has to be built locally.

Gerschtli commented 3 years ago

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

bbigras commented 3 years ago

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

Would that help with the uid thing, or it's related to the other problems I had?

The reason for the wrong uid and gid is, that these values have to be generated on the destination machine. There is a derivation built in modules/user.nix, which has to be built locally.

Could it be a good idea to have a way to optionally set those ids using the config file?

Gerschtli commented 3 years ago

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

Would that help with the uid thing, or it's related to the other problems I had?

I was refering to the comment you made "Edit: well the "no change" might not be normal. Maybe I ran the wrong one.".

Could it be a good idea to have a way to optionally set those ids using the config file?

I mean you could make these configurable, but the UX would be strange to first setup nix-on-droid the default way, gathering uid and gid, and then hardcode these values. Maybe I am just overthinking :D

Would it be possible to build everything remotely except this derivation? IIRC there aren't big dependants of this derivation, so it should be alright to build the remaining things locally on the destination phone.

bbigras commented 3 years ago

I was refering to the comment you made "Edit: well the "no change" might not be normal. Maybe I ran the wrong one.".

Ok I think this issue is working now. Thanks. I'll take a look again if I see the message again.

I mean you could make these configurable, but the UX would be strange to first setup nix-on-droid the default way, gathering uid and gid, and then hardcode these values. Maybe I am just overthinking :D

I was thinking only people wanting to deploy from a desktop like me would have to handle this.

Would it be possible to build everything remotely except this derivation? IIRC there aren't big dependants of this derivation, so it should be alright to build the remaining things locally on the destination phone.

I have no idea how I could build it on my phone while deploying from my desktop (or from github actions).

Do you think it could be done at the activation step (on the phone)?

Gerschtli commented 3 years ago

The activation step is too late, the uid and gid need to be available to build the nix-on-droid generation. Maybe we could refactor everything, but I don't it will work.

Sounds like an exciting idea to deploy nix-on-droid config via github actions. If the uid/gid part is the only thing stopping you, you could add these two values as option if you add a note in their description, when to set these manually and when you should rather leave them emtpy and using the ids.nix derivation. I would be fine with this change.

Gerschtli commented 3 years ago

And if it works for you, it would be awesome if you could and a wiki page explaining your setup :)

bbigras commented 3 years ago

Sounds like an exciting idea to deploy nix-on-droid config via github actions.

Yeah. It would work for people having access to an aarch64 build box though (which I don't). I should check if github actions has aarch64 nodes.

Right now my nix-config builds multiple machines (desktop and laptop) with github actions and pushes it to cachix. Then on my computer I deploy it. It just takes the time required to download and deploy.

If the uid/gid part is the only thing stopping you, you could add these two values as option if you add a note in their description, when to set these manually and when you should rather leave them emtpy and using the ids.nix derivation. I would be fine with this change.

Thanks I'll take a look.

And if it works for you, it would be awesome if you could and a wiki page explaining your setup :)

Will do! :smile:

Gerschtli commented 3 years ago

Unfortunately github actions are not available for aarch64-linux AFAIK..

bbigras commented 3 years ago

I'm using something like the following to build and deploy nix-on-droid with deploy-rs, but I just realized that I'm not using the nix-on-droid overlays.

Any ideas how to set the overlays?

let
pixel2 = (inputs.nix-on-droid.lib.aarch64-linux.nix-on-droid { config = ../hosts/pixel2; }).activationPackage;
in
{
{
  deploy = {
    autoRollback = true;
    magicRollback = true;
    user = "root";

    nodes = [
    {
      pixel2 = {
        hostname = "pixel2";

        # to prevent using sudo
        sshUser = "nix-on-droid";
        user = "nix-on-droid";

        profiles.nix-on-droid.path = deploy-rs.lib.aarch64-linux.activate.custom
          pixel2
          (pixel2 + "/activate");
      };
    };
];
  };
}

https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/nix/deploy.nix https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/flake.nix https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/hosts/pixel2/default.nix

Gerschtli commented 3 years ago

Unfortunately I do not know deploy-rs, but you can use the overlays directory and source all overlays via overlays/default.nix as a list.

pinage404 commented 2 months ago

Hello,

I did a deploy script before finding this issue

The script rely on SSH

geoffreygarrett commented 1 week ago

Managed deploying with deploy-rs, though I'm pretty sure the rollback isn't working exactly as intended. Feel free to add this to the wiki.

Solution for Deploying Using deploy-rs with nix-on-droid

Steps:

  1. SSH Setup

    Setup SSH on nix-on-droid if not done already. Add your desktop's SSH public key to the nix-on-droid user's ~/.ssh/authorized_keys file.

  2. UID/GID Handling

    The primary issue is ensuring the correct uid and gid for the nix-on-droid user on your phone. When building on your desktop, these might not match, leading to permission issues.

    • Find the UID and GID on your Android device:

      id nix-on-droid

      This command returns the uid and gid of the nix-on-droid user on your device.

    • Set the UID and GID explicitly in your nix-on-droid configuration:

      { 
user.uid = <uid>;
user.gid = <gid>;
} # Replace <uid> and <gid> with the values from your device

  3. Set Up the deploy-rs Configuration

    The activation comes from above in this thread itself. Here's a helper to make it a little less repetitive.

    let
 activateNixOnDroid =
   configuration:
   inputs.deploy-rs.lib.aarch64-linux.activate.custom
     configuration.activationPackage
     "${configuration.activationPackage}/activate";
in

    Here's how to configure your deploy-rs for nix-on-droid:

    deploy.nodes = {
 "pioneer" = {
   hostname = "pioneer.nixus.net"; # Replace with your device's hostname or IP (I use `dnsmaq` for local DNS)
   profiles.system = {
     sshUser = "nix-on-droid";
     user = "nix-on-droid";
     magicRollback = true;
     sshOpts = [ "-p" "8022" ]; # Adjust port if necessary (Step 1 dependent)
     path = activateNixOnDroid self.nixOnDroidConfigurations.pioneer;
   };
 };
};

    Note: The hostname pioneer.nixus.net is specific to my setup using dnsmasq for local DNS resolution. Adjust it according to your environment.

  4. Handle Multiple Devices

    If deploying to multiple devices, define specific configurations for each, especially if they have different uid and gid values.

    nixOnDroidConfigurations = {
 "pioneer" = nix-on-droid.lib.nixOnDroidConfiguration {
   pkgs = pkgsFor "aarch64-linux";
   modules = [
     ./nix/hosts/pioneer.nix
     # Include other modules like Home Manager if needed
     { 
       user.uid = 10701;
       user.gid = 10701;
     } # Replace with the UID and GID from your device
   ];
 };

 "voyager" = nix-on-droid.lib.nixOnDroidConfiguration {
   pkgs = pkgsFor "aarch64-linux";
   modules = [
     ./nix/hosts/voyager.nix
     { 
       user.uid = 10403;
       user.gid = 10403;
     } # Replace with the UID and GID from your other device
   ];
 };
};

  5. Set Up Cachix for Substitutes

    To satisfy dependencies like static-proot, ensure your Nix settings include the necessary substituters and trusted public keys. This is crucial because nix-on-droid relies on prebuilt cross-compiled binaries for proot-static, which are specified by hard-coded Nix store paths in the configuration.

    Nix Settings:

    {
 nix.settings = {
   substituters = [
     # "https://cache.nixos.org/"
     # "https://nix-community.cachix.org"
     "https://nix-on-droid.cachix.org" <----
   ];

   trusted-public-keys = [
     # "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
     # "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
     "nix-on-droid.cachix.org-1:56snoMJTXmE7wm+67YySRoTY64Zkivk9RT4QaKYgpkE=" # <---
   ];
 };
} # Pretty sure you only need what's pointed out, but I kept my full Cachix config in case I'm missing something else.

    Reason for the Cache:

    The nix-on-droid configuration specifies the prootStatic binary using hardcoded Nix store paths that point to prebuilt cross-compiled binaries:

    environment.files = {
 prootStatic =
   let
     crossCompiledPaths = {
       aarch64-linux = "/nix/store/7qd99m1w65x2vgqg453nd70y60sm3kay-proot-termux-static-aarch64-unknown-linux-android-unstable-2024-05-04";
       x86_64-linux = "/nix/store/pakj3svvw84rhkzdc6211yhc2cgvc21f-proot-termux-static-x86_64-unknown-linux-android-unstable-2024-05-04";
     };
   in
   "${crossCompiledPaths.${targetSystem}}";
};

    (Reference: nix-on-droid/modules/environment/login/default.nix#L90)

    These binaries are not built locally during deployment (I have no idea how it would be setup), so having the cache set up ensures they can be fetched from the nix-on-droid Cachix cache.

  6. Configure Overlays for nix-on-droid

    Ensure your overlays are properly set up to include nix-on-droid:

    pkgsFor = system: import nixpkgs {
 inherit system;
 config = {
   allowUnfree = true;
   overlays = [
     (import ./nix/overlays)              # Your custom overlays
     # (final: prev: { nix-on-droid = nix-on-droid.packages.${system}; }) # not necessary, I use it in one of my flake apps
   ] ++ lib.optional (custom.isAndroid system) nix-on-droid.overlays.default; 
 };
}; # isAndroid is just checking against a list of [ "aarch64-linux" ], as I believe only that is really supported here

  7. Finalize and Deploy

    With all configurations in place, deploy using:

    deploy --targets ".#pioneer" -- --impure

    Replace pioneer with the appropriate node name from your deploy.nodes configuration. (let me know if this can be done without --impure)

    Deployment Log ```bash ℹī¸ [deploy] [INFO] Evaluating flake in . trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix 🚀 ℹī¸ [deploy] [INFO] The following profiles are going to be deployed: [pioneer.system] user = "nix-on-droid" ssh_user = "nix-on-droid" path = "/nix/store/p2jbjbr1lksk2aa26cfhxlzs0rxnbfgm-activatable-nix-on-droid-generation" hostname = "pioneer.nixus.net" ssh_opts = ["-p", "8022"] 🚀 ℹī¸ [deploy] [INFO] Building profile `system` for node `pioneer` 🚀 ℹī¸ [deploy] [INFO] Copying profile `system` to node `pioneer` 🚀 ℹī¸ [deploy] [INFO] Activating profile `system` for node `pioneer` 🚀 ℹī¸ [deploy] [INFO] Creating activation waiter ⭐ ℹī¸ [activate] [INFO] Activating profile 👀 ℹī¸ [wait] [INFO] Waiting for confirmation event... Activating linkBinSh Activating linkUsrBinEnv Activating setPriorityHomeManagerPath Activating installLogin Activating installLoginInner Activating installPackages replacing old 'nix-on-droid-path' installing 'nix-on-droid-path' Activating installProotStatic Activating linkColors Activating linkFont Activating setUpEtc Activating sops-nix sops-install-secrets: Imported /data/data/com.termux.nix/files/home/.ssh/id_ed25519 as age key with fingerprint `***KEY_CENSORED***` Activating sshd Setting up OpenSSH... Authorized keys file already exists. Skipping... Setting correct permissions... OpenSSH setup complete. Activating homeManager Starting Home Manager activation Activating checkFilesChanged Activating checkLinkTargets Activating writeBoundary Activating installPackages Activating migrateGhAccounts Activating linkGeneration Cleaning up orphan links from /data/data/com.termux.nix/files/home No change so reusing latest profile generation 48 Creating home file links in /data/data/com.termux.nix/files/home Activating onFilesChange Activating reloadSystemd User systemd daemon not running. Skipping reload. Activating sops-nix User systemd daemon not running. Probably executed on boot where no manual start/reload is needed. Activating linkProfile ⭐ ℹī¸ [activate] [INFO] Activation succeeded! ⭐ ℹī¸ [activate] [INFO] Magic rollback is enabled, setting up confirmation hook... 👀 ℹī¸ [wait] [INFO] Found canary file, done waiting! ⭐ ℹī¸ [activate] [INFO] Waiting for confirmation event... 🚀 ℹī¸ [deploy] [INFO] Success activating, attempting to confirm activation 🚀 ℹī¸ [deploy] [INFO] Deployment confirmed. .dotfiles on ī˜ develop [$?] took 49s ```

    Hope this helps someone. Final note, if you're cross compiling from x86_64-linux on your nixos desktop, add this to your desktops config and switch:

    boot.binfmt.emulatedSystems = [ "aarch64-linux" ];