Androids dreaming of stormy weather

[cdecker]

There have been a couple of attempts to get c-lightning running on Android devices, and I thought it might be a good idea to pull experiences together to see what can be done to make Android support a possibility and make the process of getting it running on Android simpler.

I know that the abcore team (@greenaddress and @lvaccaro) has done some preliminary work to get c-lightning to compile on Android (https://github.com/greenaddress/bitcoin_ndk/pull/21). The approach they followed is:

• Patch the makefile, the endiannes library and the JSON-RPC
• Generate the config.vars and gen_header_versions.h on device, since the configurator runs on the compiler host
• Finally compile the dependencies and lightningd using clang for the target arch

Ideally we'd backport the patches they needed to apply in order to internalize that part. @NickeZ on the other hand has been tackling the need to pre-generate the target config.vars and gen_header_versions.h files in https://github.com/ElementsProject/lightning/pull/3464. That'd allow us to compile the configurator for the target arch and run it using the QEMU wrapper. Hopefully that eliminates the need for pre-canned "generated files".

Finally @remyers is also working on getting Android support working, with a host configurator but otherwise cross-compiling for ARM: https://github.com/ElementsProject/lightning/issues/3473

Ideally we'd pool our collective experiences in getting the cross-compiled flow working, and document it for others to find in future.

What do you guys think? Are there any stumbling stones that I am missing? Could this allow us to cross-compile reproducible builds as well?

[Saibato]

Nice, i.e. i did once run c/ln compiled in termux on the phone from old source source that i patched step by step to reach master side by side with a full node over orbot tor proxy, stopped this effort a year or so. The pruned full node i synced fast on a laptop and then insert the sd card in the phone. Then i did some patches on the core node and c/ln to not fall behind and get block info even on prunded node, i.e. if you know funding unspent that's not perfect but better than nothing. Since the power consumption is high and the old phone had a bad battery pack i stopped this and lets be honest i am sure the custodial's apps incl. BlockS. efforts would not react welcome like, if i begin to crush there BM on there own platform before moon, so i stopped work on this. We should use there VC bugs improve ln and crush them later. Hopefully others have time and can improve, and can in general reduce power consumption of c/ln !!!

[darosior]

What do you guys think?

I think that since I've started working on moving our Bitcoin backend to plugins, I've started dreaming of having C-lightning + a Bitcoin plugin hitting my esplora API englobed in a Kivy Android app :grin:. So I'm very much interested in this !

[cdecker]

Hopefully others have time and can improve, and can in general reduce power consumption of c/ln !!!

Is it really c-lightning's power consumption or is it the co-located bitcoind? In my experience c-lightning is rather lightweight, running on a shoe-string budget.

[Saibato]

Is` it really c-lightning's power consumption or is it the co-located bitcoind?

Sure more bitcoind! And c-lightning is rather lightweight, but can we ever see soverain trustless private pseudonymous LN and make up for the sins of NAT, without a full node?

And will it not be in the near future, that LN is integral part of a matching core node?

In general SW dev in my view should not only in sight of global warming have an eye on power consumption, every byte touched or moved is one byte to much. Even if some magic wizard comes up with an LN idea to have cold sign keys and offline nodes that just need a short sync to push or forward tx. There will always be big hubs or merch. that run there nodes 24/7. And i guess there will be many scenarios, where LN on there phone will be there only POS. :sweat_smile:

̶E̶D̶I̶T̶:̶ ̶S̶h̶o̶r̶t̶ ̶s̶i̶d̶e̶ ̶n̶o̶t̶e̶,̶ ̶c̶o̶l̶o̶s̶s̶u̶s̶ ̶h̶t̶t̶p̶s̶:̶/̶/̶t̶w̶i̶t̶t̶e̶r̶.̶c̶o̶m̶/̶s̶c̶i̶e̶n̶c̶e̶m̶u̶s̶e̶u̶m̶/̶s̶t̶a̶t̶u̶s̶/̶1̶2̶2̶5̶0̶2̶1̶4̶8̶7̶1̶2̶1̶3̶3̶8̶3̶7̶0̶ ̶w̶a̶s̶ ̶a̶l̶l̶e̶g̶e̶d̶l̶y̶ ̶u̶s̶e̶d̶ ̶t̶o̶d̶a̶y̶ ̶i̶n̶ ̶4̶4̶ ̶f̶o̶r̶ ̶t̶h̶e̶ ̶1̶s̶t̶ ̶t̶i̶m̶e̶,̶ ̶w̶e̶ ̶s̶h̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶a̶l̶s̶o̶ ̶a̶n̶ ̶o̶p̶e̶n̶ ̶e̶y̶e̶ ̶o̶n̶ ̶N̶u̶m̶b̶e̶r̶W̶a̶n̶g̶,̶ ̶s̶o̶ ̶s̶o̶m̶e̶t̶i̶m̶e̶s̶ ̶m̶o̶r̶e̶ ̶m̶a̶g̶i̶c̶ ̶b̶y̶t̶e̶s̶ ̶a̶r̶e̶ ̶g̶o̶o̶d̶.̶

[remyers]

I'm going to close #3473 because I can now build and run clightning on my android phone using the bitcoin_ndk project referenced in greenaddress/bitcoin_ndk#21.

My setup: MacOS Catalina 10.15.2 (Macbook Pro, 8GB RAM, 128GB HD)

1. Follow steps to install docker-machine & virtualbox (MacOS guide)

   $ brew install docker docker-machine
   $ brew cask install virtualbox
   Look for: “installer: The install was successful.”

   Note: For MacOS you need to create symlinks to sha256sum so the final report at step 7 works, but it does not prevent creation of the executable artifacts. (MacOS guide)

2. [optional] Set the location where you want the docker-machine files and certificates to be stored, if you don't have enough disk space at the default location.

   $ export MACHINE_STORAGE_PATH=/Volumes/Personal/android/machine
   $ export DOCKER_CERT_PATH=/Volumes/Personal/android/machine/certs

3. Clone the bitcond_ndk project:

   $ git clone https://github.com/greenaddress/bitcoin_ndk.git
   $ cd bitcoin_ndk
   $ git checkout cln_test

4. Create the docker-machine image using virtualbox driver and call it ‘default’. Note: I allocated 100 GB for my disk, but 20 GB is probably sufficient. I also allocated 3 GB of RAM but more is better, especially if you use num_jobs=4 instead of making the changes I made in step 6. Note: $PWD should point to the folder you cloned bitcoin_ndk into.

   $ docker-machine create --driver virtualbox --virtualbox-disk-size "100000" --virtualbox-memory "3072" --virtualbox-share-folder /Users/Users --virtualbox-share-folder $PWD:/repo default
   Look for: “Docker is up and running!”
   $ docker-machine env default
   $ eval "$(docker-machine env default)"

5. Follow the steps from the bitcoin_ndk/README.md to build the bitcoin_ndk docker image: $ docker build .

6. I made some small changes to bitcoin_ndk so it would build faster on my resource limited computer: fetchbuild.sh:53 I used 'num_jobs=2' instead of 4 run.sh:8: I used 'ARCHS="aarch64-linux-android=64" ' instead of building for every architecture run.sh:17,18: I commented out the bitcoinknots and elementsproject 'build repo ...' lines

7. The final step in bitcoin_ndk/README.md will use the docker container to build the artifacts for each target machine. $ ./run.sh

8. Unzip the tar files created for your target device and copy them to your android phone: Note: The c-lightning executables cross-compiled for my aarch64-linux-android Pixel 2 XL are only about 36MB on disk.

   $ tar -xf aarch64-linux-android_bitcoin.tar.xz 
   $ abd push aarch64-linux-android_bitcoin/* /data/local/tmp
   $ adb shell
   taimen $ cd /data/local/tmp
   taimen $ ./lightningd --version

   You should expect to get back: "v0.7.3-modded" You are using the wrong target build if you get back: "/system/bin/sh: ./lightningd: not executable: 64-bit ELF file"

I did some preliminary testing with a lightning config file on my phone which accesses a regtest bitcoind on my host machine. The bitcoin.conf on my host machine allows RPC access to connections on my local network:

# $HOME/Library/Application Support/Bitcoin/bitcoin.conf
txindex=1
regtest=1
rest=1
rpcallowip=192.168.1.0/255.255.255.0
rpcauth=foo:8526a5a90d3d7d53967dfb1920e1ade0$d88b4cc7f35abdcccd4db591626f861c2f408b28a52c483bd9e5c3827efdfb66).

I tested it like this:

$ abd shell
taimen $ cd /data/local/tmp
taimen $ mkdir regtest
taimen $ mkdir regtest/plugins
taimen $ cp plugins/* regtest/plugins
taimen $ cat << EOF > regtest/config
# file:/data/local/tmp/regtest/config
network=regtest
# daemon
log-level=debug
log-file=/data/local/tmp/regtest/log
rescan=5
bitcoin-cli=/data/local/tmp/bitcoin-cli
bitcoin-datadir=/data/local/tmp
bitcoin-rpcuser=foo
bitcoin-rpcpassword=bar
bitcoin-rpcconnect=192.168.1.20
EOF
taimen $ ./lightningd --lightning-dir=/data/local/tmp/regtest &
taimen $ ./lightning-cli --lightning-dir=/data/local/tmp/regtest listfunds
{
   "outputs": [],
   "channels": []
}

Many thanks to @lvaccaro, the bitcoin-ndk team and everyone else who has helped make cross-compiling clightning for android possible, and now even easier to do.

[lvaccaro]

Thanks for sharing @cdecker :-) @remyers there are some more fixes on bitcoin_ndk due to better support cross-compilation steps on different machine. You could find full binaries at https://github.com/lvaccaro/bitcoin_ndk/releases/tag/v0.18.1.2

I really like add a CI for cross-compilation (github or gitlab) but this requires a bit more effort to run configurator as just explain. I'll try to keep in touch :-)

[remyers]

@remyers there are some more fixes on bitcoin_ndk due to better support cross-compilation steps on different machine. You could find full binaries at

Great, I will sync the latest changes. It is really helpful to have your build scripts so I can build with some small modifications to c-lightning we are using to test making channel updates over mesh radio.

[icota]

I did some pioneering work on this 2 years ago so I'd like to chime in.

Are there any stumbling stones that I am missing?

In my experience the stumbling blocks are at runtime. c-lightning used to fork() itself quite a lot (still does?) which didn't sit right with some flavours of Android (and is a total non-starter on iOS). Essentialy the system wants to manage the lifecycle of the apps itself so any forked process is liable to get killed. This is not consistent and depends on the Android vendor, which, IMO makes it even worse.

Since Android Q use of exec() is restricted and will be removed in the future. My guess is they will restrict forking in some way as well which can be problematic for a c-lightning instance with multiple peers (subdaemons).

Sorry for being a downer. :cry:

[lvaccaro]

There are lots of painful downside( as described also in porting bitcoin core on android see Abcore project) due to exec, fork, but also background handling. Moreover classic unix socket doesn't work and it needs to use network socket or abstract namespace socket instead.

I think that cross-compiling for Android is a good feature and in the future and a prototyping app could be happily announced. Maybe this will not be the final step, but an initial step to study better mobile integration. (in the past, some apps has similar issues Termux, Tor and they try to solve in some ways.)

[ZmnSCPxj]

Ouch. We use Unix sockets because Unix sockets are magical and can transport file descriptors across processes (via ancillary data), and we pass fds around a lot between daemons. If that "doesn't work" then that is as much of a problem as running C-Lightning on Windows: https://github.com/ElementsProject/lightning/issues/1628#issuecomment-500185142

[cdecker]

Admittedly there is an API for fd passing on Windows, it's just a bit more convoluted than with Unix sockets. I'm still convinced that given enough time to write a shim we can run on Windows without any architectural changes.

Sadly the same cannot be said if future Android versions start restricting fork and execve. In that case we'd need to have a monolithic build in which we replace processes with threads. They'd still be communicating with socketpairs to maintain the current semantics, but from the outside it looks just like a single process. This is a very simplistic view, and I'm sure there are plenty of pitfalls to actually implementing a monolithic compilation target, but I'm sure it can be done.

[remyers]

Thanks for the links and review of the situation @icota. To summarize, it sounds like exec() and fork() will run on currently allowed Android APIs but that there is no guarantees that long running exec/fork processes will not be killed off randomly by the phone OS. That it works at all is something that will be phased out in Android and is already disallowed in iOS.

From what I understand, the proper way to manage process lifecycles on mobile is to access native code as a library via JNI or equivalent and manage daemons as service processes.

Creating a monolithic target as @cdecker describes is one approach, but I wonder if there isn't a way to keep the independent daemon model, but replace exec/fork to instead use a system compatible with the the Android and iOS process lifecycle model.

Perhaps this is too simplistic, but I would want to see if instead of exec() and fork() we can create Android service processes for lightningd and it's daemons. Each service would run the main() entry point in a thread. When prompted by the Android lifecycle system, the service process would make JNI calls to cleanly shutdown or restart each native service/process.

[ZmnSCPxj]

Depends on how heavyweight an Andriod service is. We launch a process for each channel, and for major state changes of the channel (e.g. transiioning from "negotiating a new channel with peer" to "normal channel operation", or transitioning from "normal operation" to "negotiating the mutual close feerate", for example) we stop one process and then start a replacement to handle the channel.

This is fine in GNU/Linux since processes are as lightweight as threads (threads are processes in GNU/Linux, that happen to have shared virtual memory mappings of the same parts of the swapfile). If Android services are heavier, then we would be operating against the grain.

[darosior]

FWIW I was also able to build and run C-lightning for my non rooted Android phone. I've put up a branch for only building C-lightning (actually what @remyers did too), and only for armv7a (which I think is the most portable ?):

git clone -b cln_only https://github.com/darosior/bitcoin_ndk/
cd bitcoin_ndk && docker build . && ./run.sh

I'm now trying to port @lvaccaro 's patches to v0.8.1(rc3) but there are still a few quirks..

Update: I could succesfully build the freshly released v0.8.1 for armv7a-linux-androideabi=32. The patches are available here:

git clone -b clightningv0.8.1 https://github.com/darosior/bitcoin_ndk && cd bitcoin_ndk && docker build . && ./run.sh

If you want to build for another architecture, change the ARCH in run.sh. If you want to build evrything for every architecture see https://github.com/greenaddress/bitcoin_ndk/pull/22 .

[remyers]

RE: using an Android service vs. Linux thread, I think a key question is if it makes sense architecturally in clightning to abstract the fork() call currently used to launch subdaemons. How hairy would it be to use a JNI callback to launch subprocesses in an Android friendly way?

Perhaps it is cleaner (and more lightweight) to spawn threads as @cdecker suggested.