build.sh script improvements

[Gerschtli]

I had another look at your build.sh while trying to use it and I feel like it is overly complicated. If you don't want to test the bootstrap zip itself, you can always chose to use github etc. for channel/flake urls. If you want to test bootstrap zip, you need a webserver anyway, so why not always push to the webserver. That way you can always just run this little script.

Do I miss something or does this script already cover all cases to help the deployment?

Note: This script only works with https://github.com/t184256/nix-on-droid/pull/204 but I didn't want to start this discussion in this PR.

#!/usr/bin/env bash
set -euo pipefail

cd "$(dirname "$0")"

DEST_URL="$1"     # like https://example.com/bootstrap
RSYNC_TARGET="$2" # like user@host:/path/to/bootstrap

SOURCE_FILE="source.tar.bz2"

function build_and_get_store_path() {
    nix build --no-link --json "$@" \
        | jq --raw-output '.[0].outputs.out'
}

echo "building proot..."
PROOT="$(build_and_get_store_path ".#prootTermux")"

PROOT_HASH_FILE="modules/environment/login/default.nix"
echo "patching proot path in $PROOT_HASH_FILE..."
grep "prootStatic = \"/nix/store/" "$PROOT_HASH_FILE"
sed -i "s|prootStatic = \"/nix/store/.*\";|prootStatic = \"$PROOT\";|" "$PROOT_HASH_FILE"
grep "prootStatic = \"/nix/store/" "$PROOT_HASH_FILE"

echo "building bootstrapZip..."
export NIX_ON_DROID_CHANNEL_URL="$DEST_URL/$SOURCE_FILE"
export NIX_ON_DROID_FLAKE_URL="$DEST_URL/$SOURCE_FILE"
BOOTSTRAP_ZIP="$(build_and_get_store_path --impure ".#bootstrapZip")"

echo "creating tar ball..."
tar \
    --exclude=".git" \
    --exclude="$SOURCE_FILE" \
    --transform 's,^,nix-on-droid/,' \
    --create \
    --bzip2 \
    --file "$SOURCE_FILE" \
    .

echo "uploading artifacts..."
rsync --progress \
    "$SOURCE_FILE" \
    "$BOOTSTRAP_ZIP/bootstrap-aarch64.zip" \
    "$RSYNC_TARGET"

[t184256]

I agree that for testing channel changes with no bootstrap changes it's enough to push to git and adjust channels, while for testing bootstrap changes with no channel changes it's enough to serve a bootstrap zipball somewhere.

This script caters to the case where you have changes that span both zipball and channels. As the android app doesn't offer UI for specifying custom channel URLs similarly to bootstrap URLs (should it?), I was testing such changes by baking custom bootstrap zipballs with urls inside them pointing to feature branch remotes, autodetected from the branch I'm currently on.

[Gerschtli]

I am not sure if I am getting your point. Let me try to rephrase what you wrote:

If I want to change stuff in the channel/flake only (no bootstrap changes), I would then create a bootstrap zip ball with my custom channel/flake and boot from that one? Why not call

nix-channel --add http://... nix-on-droid
nix-channel --update nix-on-droid

on device instead? If you really want to do some changes to the channel that is affecting bootstrap process then you are in a test case scenario that spans both zipball and channels.

[t184256]

"Bootstrap-only" cases are probably something nobody cares about and can be treated a subset of "change both".

"Channel-only" case:

If you want to change stuff in the channel/flake only (no bootstrap changes), you can bootstrap with a vanilla zipball and execute commands on device. No need for build.sh, really.

"Change both" case:

If you really want to do some changes to the channel that is affecting bootstrap process then you are in a test case scenario that spans both zipball and channels.

Yes, and that's the generic problem I was solving with build.sh

Sub-usecases for it could be:

• "I don't trust this guy's zipballs" --- can be replaced with documentation instead
• "How do you even build this stuff" --- can be replace with documentation instead
• "I want to quickly test a PR on a real device": gh pr checkout 204 && rsync_to=my_server:/path/ ./build.sh --- can be replaced with an out-of-tree tool

[Gerschtli]

So basically there are two options on how to test things:

1. "channel-only": bootstrap with vanilla and change channel on device after bootstrap
2. "change both": bootstrap with custom zip ball and custom channel

Case 1 does not need any script, you could just use a push to github or you can use the script to push a tar ball to a webserver if you have it set up already to not push every change before testing.

Case 2 needs a script for bootstrap zip ball creation and deployment. For the channel, the same options are available as for case 1 but you need a webserver to use your bootstrap zip ball so there is no benefit in using github branches for it.

For you sub-use cases: We lack a lot of documentation here but in my opinion a complex shell script is a bad way to document stuff. I think a tool for quickly updating proot store paths, building bootstrapZip, packing a channel tar and uploading it somewhere, is either way a good thing to have in this repository.

How about the following:

• We add/improve documentation for building, testing (fakedroid and stuff), and deploying things and its requisites (like a webserver or a git repo)
• Improve/simplify the build/deploy script into a flake output nix run .#buildAndDeploy or similar with nice CLI args

That should clarify potential misunderstandings/trust problems while helping us and other contributors for easier development.

[t184256]

Hm, didn't think about having a deploy tool in a flake! I can totally see myself using it every time I work on n-o-d.

I'm fine with deploying channel/flake tarballs together with zipballs instead of pushing WIP branches to github.

If we go that way, sounds like the most important inputs would be some $PUBLIC_URL and $RSYNC_TARGET, while core of it would boil down to

rsync $(env NIX_ON_DROID_CHANNEL_URL=$PUBLIC_URL/$(basename $CHANNEL_TARBALL) build '.#bootstrapZip' --impure --print-out-paths) $CHANNEL_TARBALL $RSYNC_TARGET/

Doesn't have to have novels full of text in it =)

[Gerschtli]

Oh didn't know about --print-out-paths, that will be helpful :)

I will see what I can come up with for this tool and the docs :)

[Gerschtli]

--print-out-paths does not seem to exist, where did you find it?

[t184256]

man nix3-build / nix build --help, nix 2.11.0