Enhancement: Build instructions for Android

[fabian-z]

I was successful in statically compiling the Pond CLI client for Android, so I thought I'll share my steps here for review before submitting a pull request. Looking forward to your feedback :smiley:

• Prequisites on build computer: android sdk, mercurial, git, gcc with working build environment
• Prequisites on Android: root access, enabled USB debugging & root access for adb, Orbot, Terminal Emulator
• Recommended on Android: BusyBox (with bash), Hacker's Keyboard (5-row pc layout used from bash makes vim more usable. You will need to start bash first and pond afterwards)
• All Apps can/should be get from F-Droid official repos.

Remarks:

• I used Arch Linux (x64) for building and CyanogenMod 11 with Android 4.4.4 (ARMv7) for running, but the directions should roughly apply for other architectures/distributions/ROMs/Android versions too.
• Replace $HOME/pond-android with your chosen path.
• For now you will have to copy your distributions ca-certificates.crt to your device, as access to the Android CA store requires Java Keystore code.
• If your Android ROM doesn't feature vim (e.g. stock ROM) or it fails to start (like it is the case for recent CM), you can get it here: http://gdr.geekhood.net/gdrwpl/vim-android.php Just extract data/local/bin/vim from the archive and put it on your device as /system/xbin/vim.
• For some devices (mostly HTC & Samsung), it may not be possible to directly write to /system from the booted ROM. Instead you may have to do step 7 from recovery as root.
• Please note that the Pond statefile will be stored in /data/data/jackpal.androidterm/app_HOME/.config/pond or /sdcard/.config/pond. As with every non-TPM version of Pond, secure erasure cannot be guaranteed, so be sure to at least encrypt your Android!

1. Choose a working directory under which everything will be downloaded and installed.

% mkdir $HOME/pond-android && cd $HOME/pond-android

2. Get Go source code and build with tests 

% hg clone -u release https://code.google.com/p/go
% cd go/src
% ./all.bash

3. Get cross compilation helper scripts from GitHub.

% git clone git://github.com/davecheney/golang-crosscompile.git
% source golang-crosscompile/crosscompile.bash

4. Add Go binary to your path (if not already done). Be sure to use the same version of Go
you just compiled. This could conflict if you have another version installed manually or
via package managers. Build go cross-compiler for every architecture.

% export PATH=$PATH:$HOME/pond-android/go/bin
% go-crosscompile-build-all

5. Create gopkg folder and set environment variables GOPATH and GOARM
To determine which value for GOARM you need look here:
https://code.google.com/p/go-wiki/wiki/GoArm

% mkdir $HOME/pond-android/gopkg && export GOPATH=$HOME/pond-android/gopkg && export GOARM=7

6. Get current Pond source and dependencies & build for ARM

% go-linux-arm get -tags "nogui notpm" github.com/agl/pond/client

7. You will find your binary in $HOME/pond-android/gopkg/bin/linux-arm/.
Copy to device via adb, additionally copy ca-certificates.crt file to avoid error because
of missing root certificates. We will temporarily store the files on /sdcard, moving them
to the appropriate places using the adb shell. You obviously need your device plugged
into USB and have USB debugging allowed with root access for the next steps. You will
also need to create a tmpfs mount here and make it permanent with an init script
(the init script step may vary across android versions and roms!).

% adb push $HOME/pond-android/gopkg/bin/linux-arm/client /sdcard/
% adb push /etc/ssl/certs/ca-certificates.crt /sdcard/
% adb shell
shell@xxx:/ $ su
root@xxx:/ $ mount -o rw,remount /system
root@xxx:/ $ mkdir -p /etc/ssl/certs
root@xxx:/ $ cp /sdcard/ca-certificates.crt /etc/ssl/certs/ && chmod 444 /etc/ssl/certs/ca-certificates.crt
root@xxx:/ $ cp /sdcard/client /system/xbin/pond && chmod 755 /system/xbin/pond
root@xxx:/ $ mkdir /data/local/tmpfs && chmod 777 /data/local/tmpfs
root@xxx:/ $ mount -o size=64M -t tmpfs tmpfs /data/local/tmpfs
root@xxx:/ $ echo -e  "#!/system/bin/sh\nmount -o size=64M -t tmpfs tmpfs /data/local/tmpfs" > /etc/init.d/11tmpfs
root@xxx:/ $ chmod 755 /etc/init.d/11tmpfs

8. Start Orbot App and establish a Tor circuit. Afterwards start Terminal Emulator and startup pond! Note that Pond won't work over adb shell.

And finally, if you trust me, you can try out my prebuilt ARMv7 binary: http://d-h.st/DfL

SHA256 Hash: f3ffb370389058c718d96e11e140a4a43e281718688db19c8245cf40f5c0465b
MD5 Hash: 301e732dbb9b6ff20347d4da7cffe817

Note that you will still need to follow step 7 (with your own paths of course) and possibly install vim on your device for Pond to work.

[fabian-z]

Screenshot:

[burdges]

Very cool thank you! :)

[ioerror]

This is fantastic - it really makes me want to put Pond into Debian - it would be great to get Pond with Lil'Debbie.

[ebfe]

As I have not found a way to apply two build tags at the same time, one will have to edit some go files.

go build -tags "nogui notpm" (?)

[fabian-z]

Thank you everyone. Since I don't usually do much terminal work with my device, Lil'Debi would have been overkill for me in terms of space and complexity - the static binary does very well in these cases. Installing Pond on the other hand would be easier on Lil'Debi for now.

@ebfe Thanks for your suggestion. It actually seems to work when used directly with go (compiling for the current host). Unfortunately when cross-compiling, I run

go-linux-arm get -tags "nogui notpm" github.com/agl/pond/client and get the following error - nothing is built: package notpm: unrecognized import path "notpm"

If I use go-linux-arm build -tags "nogui notpm" instead of go-linux-arm get the error is can't load package: package notpm: cannot find package "notpm" in any of: $GOROOT, $GOPATH

Of course, If I put the tags the other way round, the error message is for nogui instead of notpm - otherwise it stays the same.

I tried serveral methods on my first compile (seperated with commas, with whitespace, with and without single or double quotation marks, etc.), but none of them yielded anything but an error.

[ebfe]

@ebfe Thanks for your suggestion. It actually seems to work when used directly with go (compiling for the current host). Unfortunately when cross-compiling, I run

go-linux-arm get -tags "nogui notpm" github.com/agl/pond/client and get the following error - nothing is built: package notpm: unrecognized import path "notpm"

This looks like a bug in golang-crosscompile. There is a fix for it that never got merged davecheney/golang-crosscompile#13.

[kpcyrd]

@ioerror

This is fantastic - it really makes me want to put Pond into Debian

yes, please!

[fabian-z]

@ebfe Nice discovery!

It actually seems like I used the predecessor of the nowadays more widely used https://github.com/laher/goxc - but nonetheless there have been some recent commits to golang-crosscompile. Since goxc is supposed to crosscompile to all platforms at once, it maybe more useful to simply fork golang-crosscompile and apply the pending pull request. Another possibility is to try and use goxc to compile only for linux/arm. I'll look into this a little further.

• Since the PR got accepted I was able to update my instructions above and keep using golang-crosscompile. Thanks @ebfe!

[burdges]

Appears Go 1.4 offers a GOOS=android option which might simplify this build process (examples)

[fabian-z]

I don't feel like it will greatly simplify the process of getting the CLI to run on an Android phone as it most probably makes no difference regarding issues with tmpfs and certificates store, which make up for most of the manual work in the process.

It could allow us to skip choosing the GOARM variable and just use GOOS=android to build a static binary, but judging from a quick google search there seem to be some issues with cgo left on this route. Well, let's wait for the dust to settle.

What this should allow for is - after ironing out the tmpfs and certificates store issues - to build pond as a library to use with a Java UI (given the Java -> Go language binding generator).

Alternatively, there is the possibility to build a simple localhost web server with a responsive HTML5/JS UI into pond (maybe having a webgui tag in addition to nogui).

I actually like the web server idea, because it would allow to package pond (with a simple webview) for Android. Comments on reddit suggest this should be possible. It would also allow for alternate GUI usage on unsupported, older distributions and, in the long run, could make the base for pond to be used from multiple access points over a network. Using HTTPS and the statefile password as user auth, this should imho provide reasonable security, too.

Any thoughts on this?

At 27.11.2014 on 01:29 Jeff Burdges wrote:

Appears Go 1.4 offers a GOOS=android option https://www.reddit.com/r/golang/comments/2kv0s1/go_14_on_android which might simplify this process (examples https://github.com/MarinX/godroid)

— Reply to this email directly or view it on GitHub https://github.com/agl/pond/issues/119#issuecomment-64730208.

[bnagy]

@fabian-z What we did is shim the core and provide an RPC API then use a native Java app. You might find you need more communication with the android runtime than you thought.

The certs issue is easily solved with a patch to just hardcode the few external CAs that panda needs - I should be able to upstream this if it will help ( but it's trivial ) [UPDATE: This is not required from 1.4, they handle Android's cert dir]

I've never had tmpfs issues ( unless I solved them very early and forgot )

I think the web server idea is a very, very bad one. It would 'work' but it creates an appalling hornets nest of extra timing / linkability / complexity and ( for multiple access points ) state issues. I realised at one point I had made this trivial so I refactored everything to make it harder again and burnt the evidence. ;)

[charlesay]

I could build a binary for CM11 on Ubuntu 14.04 and could create an account, but when I execute pond on my device the process gets killed shortly after loading the status. dmesg shows output like this: select 5370 (pond), adj 0, size 55123, to kill send sigkill to 5370 (pond), adj 0, size 55123

Can anyone reproduce this?

[burdges]

Just another tool worth keeping an eye on : https://github.com/google/gxui