denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97.94k stars 5.39k forks source link

Cross compile deno for aarch64-linux-android #19759

Open secext2022 opened 1 year ago

secext2022 commented 1 year ago

The following content is outdated, please see the new comment for a simple new method.


I find a way to manually cross compile deno 1.35.0 for aarch64 android.

It's not good, but it works:

> cd target/aarch64-linux-android/release
> ls -l deno
-rwxr-xr-x 1 s2 s2 84005408 Jul  8 03:08 deno*

> adb push deno /data/local/tmp/deno-test/5
deno: 1 file pushed, 0 skipped. 32.9 MB/s (84005408 bytes in 2.438s)
> adb shell
violet:/ $ cd /data/local/tmp/deno-test/5

violet:/data/local/tmp/deno-test/5 $ ./deno --version
deno 1.35.0 (release, aarch64-linux-android)
v8 11.6.189.7
typescript 5.1.6

violet:/data/local/tmp/deno-test/5 $ export HOME=$(pwd)
violet:/data/local/tmp/deno-test/5 $ ./deno
Deno 1.35.0
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> 0.1 + 0.2
0.30000000000000004
> Deno.version
{ deno: "1.35.0", v8: "11.6.189.7", typescript: "5.1.6" }
>

violet:/data/local/tmp/deno-test/5 $ ./deno --unstable
Deno 1.35.0
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> let a = await Deno.openKv("2.db");
undefined
> await a.set(["test"], 666);
{ ok: true, versionstamp: "00000000000000010000" }
> await a.get(["test"]);
{ key: [ "test" ], value: 666, versionstamp: "00000000000000010000" }
> 
violet:/data/local/tmp/deno-test/5 $ ls -l 2.db
-rw-r--r-- 1 shell shell 32768 2023-07-08 03:20 2.db
violet:/data/local/tmp/deno-test/5 $ 

Build on Android is not possible, so you must do cross compile. The current problem of deno for cross compile is, snapshot.

I copy code from build.rs with a little modification, and build it as a binary deno-mksnapshot. Then run deno-mksnapshot on Android device, get the output RUNTIME_SNAPSHOT.bin, COMPILER_SNAPSHOT.bin, and CLI_SNAPSHOT.bin files. Finally disable build.rs on host, and build deno.

I have to modify a lot of source files for it to compile, and I have to disable deno_ffi and ICU.

Detail build steps and patch for source code will be posted later.

secext2022 commented 1 year ago

related issues: #3839, #10613, #13948, #16965, #19399

secext2022 commented 1 year ago

I did a very simple test by running code from https://examples.deno.land/ It seems that the examples work ok.

But deno fresh can not work (Error: Unsupported platform: aarch64-linux-android):

violet:/data/local/tmp/deno-test/5 $ export HOME=$(pwd)
violet:/data/local/tmp/deno-test/5 $ export PATH=$(pwd):$PATH
violet:/data/local/tmp/deno-test/5 $ deno run -A -r https://fresh.deno.dev

  🍋 Fresh: the next-gen web framework.  

Project Name [fresh-project] 
Let's set up your new Fresh project.

Fresh has built in support for styling using Tailwind CSS. Do you want to use this? [y/N] y
Do you use VS Code? [y/N] n
The manifest has been generated for 3 routes and 1 islands.

Project initialized!

Enter your project directory using cd fresh-project.
Run deno task start to start the project. CTRL-C to stop.

Stuck? Join our Discord https://discord.gg/deno 

Happy hacking! 🦕
violet:/data/local/tmp/deno-test/5 $ cd fresh-project/
violet:/data/local/tmp/deno-test/5/fresh-project $ ls -l
total 36
-rw-rw-rw- 1 shell shell  363 2023-07-08 11:33 README.md
drwxrwxrwx 2 shell shell 3488 2023-07-08 11:33 components
-rw-rw-rw- 1 shell shell  767 2023-07-08 11:33 deno.json
-rwxrwxrwx 1 shell shell  130 2023-07-08 11:33 dev.ts
-rw-rw-rw- 1 shell shell  604 2023-07-08 11:33 fresh.gen.ts
drwxrwxrwx 2 shell shell 3488 2023-07-08 11:33 islands
-rw-rw-rw- 1 shell shell  454 2023-07-08 11:33 main.ts
drwxrwxrwx 3 shell shell 3488 2023-07-08 11:33 routes
drwxrwxrwx 2 shell shell 3488 2023-07-08 11:33 static
-rw-rw-rw- 1 shell shell  111 2023-07-08 11:33 twind.config.ts
violet:/data/local/tmp/deno-test/5/fresh-project $ deno task start
Task start deno run -A --watch=static/,routes/ dev.ts
Watcher Process started.
The manifest has been generated for 3 routes and 1 islands.

 🍋 Fresh ready 
    Local: http://localhost:8000/

An error occurred during route handling or page rendering. Error: Unsupported platform: aarch64-linux-android
    at install (https://deno.land/x/esbuild@v0.17.19/mod.js:1843:11)
    at https://deno.land/x/esbuild@v0.17.19/mod.js:1852:29
    at ensureServiceIsRunning (https://deno.land/x/esbuild@v0.17.19/mod.js:1975:7)
    at Module.initialize (https://deno.land/x/esbuild@v0.17.19/mod.js:1743:9)
    at initEsbuild (https://deno.land/x/fresh@1.2.0/src/build/esbuild.ts:120:19)
    at EsbuildBuilder.build (https://deno.land/x/fresh@1.2.0/src/build/esbuild.ts:40:13)
    at ServerContext.#buildSnapshot (https://deno.land/x/fresh@1.2.0/src/server/context.ts:402:31)
    at Object.default (https://deno.land/x/fresh@1.2.0/src/server/context.ts:772:49)
    at handler (https://deno.land/x/fresh@1.2.0/src/server/router.ts:177:34)
    at Object.next (https://deno.land/x/fresh@1.2.0/src/server/context.ts:436:34)
secext2022 commented 1 year ago

Build steps

1 Build environment setup

2 Build and run deno-mksnapshot-runtime

Source code files and patch see here: TODO

Now we get RUNTIME_SNAPSHOT.bin file.

3 Build and run deno-mksnapshot

Source code files and patch see here: TODO

Now we get COMPILER_SNAPSHOT.bin and CLI_SNAPSHOT.bin files.

4 Build deno !!

Now we get deno !

secext2022 commented 1 year ago

Modified source code files are here. And the patch diff file is here.

Modified file list:

> find . -type f
./deno-1.35.0/mksnapshot/main.rs
./deno-1.35.0/Cargo.toml
./deno_broadcast_channel-0.104.0/Cargo.toml
./deno_broadcast_channel-0.104.0/lib.rs
./deno_cache-0.42.0/Cargo.toml
./deno_cache-0.42.0/lib.rs
./deno_console-0.110.0/Cargo.toml
./deno_console-0.110.0/lib.rs
./deno_core-0.194.0/Cargo.toml
./deno_core-0.194.0/runtime/jsruntime.rs
./deno_crypto-0.124.0/Cargo.toml
./deno_crypto-0.124.0/lib.rs
./deno_fetch-0.134.0/Cargo.toml
./deno_fetch-0.134.0/lib.rs
./deno_fs-0.20.0/Cargo.toml
./deno_fs-0.20.0/std_fs.rs
./deno_http-0.105.0/Cargo.toml
./deno_io-0.20.0/Cargo.toml
./deno_kv-0.18.0/Cargo.toml
./deno_napi-0.40.0/Cargo.toml
./deno_net-0.102.0/Cargo.toml
./deno_net-0.102.0/lib.rs
./deno_node-0.47.0/Cargo.toml
./deno_node-0.47.0/lib.rs
./deno_ops-0.72.0/Cargo.toml
./deno_runtime-0.118.0/mksnapshot/main.rs
./deno_runtime-0.118.0/Cargo.toml
./deno_runtime-0.118.0/js.rs
./deno_runtime-0.118.0/js/90_deno_ns.js
./deno_runtime-0.118.0/lib.rs
./deno_runtime-0.118.0/ops/os/mod.rs
./deno_runtime-0.118.0/ops/os/sys_info.rs
./deno_runtime-0.118.0/permissions/mod.rs
./deno_runtime-0.118.0/web_worker.rs
./deno_runtime-0.118.0/worker.rs
./deno_tls-0.97.0/Cargo.toml
./deno_url-0.110.0/Cargo.toml
./deno_url-0.110.0/lib.rs
./deno_web-0.141.0/Cargo.toml
./deno_web-0.141.0/lib.rs
./deno_webidl-0.110.0/Cargo.toml
./deno_websocket-0.115.0/Cargo.toml
./deno_websocket-0.115.0/lib.rs
./deno_webstorage-0.105.0/Cargo.toml
./deno_webstorage-0.105.0/lib.rs
./fastwebsockets-0.3.1/Cargo.toml
./fastwebsockets-0.3.1/src/frame.rs
./fastwebsockets-0.3.1/src/lib.rs
./serde_v8-0.105.0/Cargo.toml
./v8-0.74.1/Cargo.toml
./v8-0.74.1/src/icu.rs
sigmaSd commented 1 year ago

Can this be done for arm32 as well ? the comments here https://github.com/denoland/deno/issues/2295 says that rustyv8 can be compiled to arm32

secext2022 commented 1 year ago

@sigmaSd

I can not compile rusty_v8 for arm32 Android.

> cd v8-0.74.1
> rustup target add armv7-linux-androideabi
> V8_FROM_SOURCE=1 cargo build -vv --release --target armv7-linux-androideabi

then get librusty_v8_release_armv7-linux-androideabi.a

> export RUSTY_V8_MIRROR=/home/s2/lib_v8
> export PATH=/opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
> export CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=armv7a-linux-androideabi28-clang
> cargo build --example hello_world --release --target armv7-linux-androideabi

   Compiling v8 v0.74.1 (/home/s2/v8-0.74.1)
error: linking with `armv7a-linux-androideabi28-clang` failed: exit status: 1
  |

  = note: ld: error: /home/s2/v8-0.74.1/target/armv7-linux-androideabi/release/deps/libv8-a84fab6083ee5863.rlib(binding.o) is incompatible with armelf_linux_eabi
          clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

error: could not compile `v8` (example "hello_world") due to previous error

I can not find a working linker.

sigmaSd commented 1 year ago

Thanks for testing I'll investigate that part

sigmaSd commented 1 year ago

this works for me

wget https://dl.google.com/android/repository/android-ndk-r25c-linux.zip
mkdir ndk && mv android-ndk-r25c-linux.zip ndk && cd ndk && unzip android-ndk-r25c-linux.zip
rm rust-toolchain.toml
rustup target add armv7-linux-androideabi
cargo install cargo-ndk
V8_FROM_SOURCE=1 ANDROID_NDK_HOME=$PWD/ndk/android-ndk-r25c cargo ndk  -t armeabi-v7a b --release -vv
secext2022 commented 1 year ago

@sigmaSd
you can try build hello_world example of rusty_v8, and run it on Android device.

I can not run it with this error:

violet:/data/local/tmp/deno-test/6 $ ./hello_world
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `256`,
 right: `248`', src/isolate.rs:559:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'assertion failed: !annex_ptr.is_null()', src/isolate.rs:656:5
stack backtrace:
   0:  0x3524e0a - <unknown>
   1:  0x3535ec6 - <unknown>
   2:  0x35232cc - <unknown>
   3:  0x3524cbc - <unknown>
   4:  0x3525a0c - <unknown>
   5:  0x3525780 - <unknown>
   6:  0x3525f0c - <unknown>
   7:  0x3525d7a - <unknown>
   8:  0x3525152 - <unknown>
   9:  0x3525be8 - <unknown>
  10:  0x3535488 - <unknown>
  11:  0x35354ea - <unknown>
  12:  0x2abeeda - <unknown>
  13:  0x2abe9a0 - <unknown>
  14:  0x2aba9e8 - <unknown>
  15:  0x2abc282 - <unknown>
  16:  0x2abc260 - <unknown>
  17:  0x3521ab8 - <unknown>
  18:  0x2abc248 - <unknown>
  19: 0xf1420a84 - <unknown>
thread panicked while panicking. aborting.
Aborted 
134|violet:/data/local/tmp/deno-test/6 $ 
sigmaSd commented 1 year ago

I see I get the same error when compiling the example

ld: error: /workspace/rusty_v8/target/armv7-linux-androideabi/release/deps/libv8-bd7ba9c626a81de1.rlib(binding.o) is incompatible with armelf_linux_eabi
          clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
secext2022 commented 1 year ago

We need to modify rusty_v8/build.rs file to add arm32 android support:

> git diff build.rs
diff --git a/v8-0.74.1/build.rs b/v8-0.74.1/build.rs
index 94e8950..57ab10f 100644
--- a/v8-0.74.1/build.rs
+++ b/v8-0.74.1/build.rs
@@ -144,7 +144,7 @@ fn build_v8() {
     let clang_base_path = clang_download();
     gn_args.push(format!("clang_base_path={:?}", clang_base_path));

-    if cfg!(target_os = "android") && cfg!(target_arch = "aarch64") {
+    if cfg!(target_os = "android") && (cfg!(target_arch = "aarch64") || cfg!(target_arch = "armv7")) {
       gn_args.push("treat_warnings_as_errors=false".to_string());
     }
   }
@@ -179,6 +179,13 @@ fn build_v8() {
       maybe_install_sysroot("arm64");
       maybe_install_sysroot("amd64");
     };
+    // armv7-linux-androideabi
+    if target_triple == "armv7-linux-androideabi" {
+      gn_args.push(r#"target_cpu="arm""#.to_string());
+      //gn_args.push("use_sysroot=true".to_string());
+      //maybe_install_sysroot("arm64");
+      //maybe_install_sysroot("amd64");
+    };

     if target_triple == "aarch64-linux-android" {
       gn_args.push(r#"v8_target_cpu="arm64""#.to_string());
@@ -207,6 +214,13 @@ fn build_v8() {
         &format!("{}/catapult.git", CHROMIUM_URI),
       );
     };
+    // armv7-linux-androideabi
+    if target_triple == "armv7-linux-androideabi" {
+      gn_args.push(r#"v8_target_cpu="arm""#.to_string());
+      gn_args.push(r#"target_os="android""#.to_string());
+
+      gn_args.push("treat_warnings_as_errors=false".to_string());
+    };
   }

   if target_triple.starts_with("i686-") {
@@ -564,13 +578,13 @@ fn find_compatible_system_clang() -> Option<PathBuf> {
 fn clang_download() -> PathBuf {
   let clang_base_path = build_dir().join("clang");
   println!("clang_base_path {}", clang_base_path.display());
-  assert!(Command::new(python())
-    .arg("./tools/clang/scripts/update.py")
-    .arg("--output-dir")
-    .arg(&clang_base_path)
-    .status()
-    .unwrap()
-    .success());
+  // assert!(Command::new(python())
+  //   .arg("./tools/clang/scripts/update.py")
+  //   .arg("--output-dir")
+  //   .arg(&clang_base_path)
+  //   .status()
+  //   .unwrap()
+  //   .success());
   assert!(clang_base_path.exists());
   clang_base_path
 }
secext2022 commented 1 year ago

Now deno can build automate in CI (github action) for aarch64-linux-android, with help from termux-docker (qemu aarch64).

https://github.com/fm-elpac/v8-src/tree/deno-android-qemu-docker https://github.com/fm-elpac/v8-src/actions/runs/5513590007

duncanmak commented 1 year ago

Wow, very inventive technique!

secext2022 commented 1 year ago

A new method to cross build deno for android:

With a modified cargo and termux-docker (QEMU), we can run build.rs on target (aarch64-linux-android).

So we do not need to modify a lot of source code files, build.rs and snapshot should just work. And it is fast (about 30 minutes on github actions CI).

This method is much simple than the old method.

see here: https://github.com/fm-elpac/v8-src/tree/deno-1.36

secext2022 commented 1 year ago

Here is the patch for the new method:

diff '--color=auto' -ru -x Cargo.lock workdir-1/deno-1.36.1/build.rs workdir/deno-1.36.1/build.rs
--- workdir-1/deno-1.36.1/build.rs  2023-08-21 13:43:45.462849697 +0800
+++ workdir/deno-1.36.1/build.rs    2023-08-21 20:17:19.569271010 +0800
@@ -340,7 +340,6 @@
       deno_broadcast_channel::InMemoryBroadcastChannel::default(),
       false, // No --unstable.
     ),
-    deno_ffi::deno_ffi::init_ops::<PermissionsContainer>(false),
     deno_net::deno_net::init_ops::<PermissionsContainer>(
       None, false, // No --unstable.
       None,
@@ -400,12 +399,9 @@
   // Host snapshots won't work when cross compiling.
   let target = env::var("TARGET").unwrap();
   let host = env::var("HOST").unwrap();
-  if target != host {
-    panic!("Cross compiling with snapshot is not supported.");
-  }

   let symbols_path = std::path::Path::new("napi").join(
-    format!("generated_symbol_exports_list_{}.def", env::consts::OS).as_str(),
+    format!("generated_symbol_exports_list_{}.def", "linux").as_str(),
   )
   .canonicalize()
   .expect(
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno-1.36.1/Cargo.toml workdir/deno-1.36.1/Cargo.toml
--- workdir-1/deno-1.36.1/Cargo.toml    2023-08-21 13:43:45.405846998 +0800
+++ workdir/deno-1.36.1/Cargo.toml  2023-08-21 18:15:47.361783037 +0800
@@ -385,3 +385,12 @@

 [target."cfg(windows)".build-dependencies.winres]
 version = "=0.1.12"
+
+[patch.crates-io]
+v8 = { path = "../v8-0.75.0" }
+deno_runtime = { path = "../deno_runtime-0.123.0" }
+deno_core = { path = "../deno_core-0.200.0" }
+serde_v8 = { path = "../serde_v8-0.111.0" }
+libz-ng-sys = { path = "../libz-ng-sys-1.1.12" }
+deno_fs = { path = "../deno_fs-0.25.0" }
+deno_node = { path = "../deno_node-0.52.0" }
Only in workdir/deno-1.36.1: target
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_core-0.200.0/Cargo.toml workdir/deno_core-0.200.0/Cargo.toml
--- workdir-1/deno_core-0.200.0/Cargo.toml  2023-08-21 14:23:37.195216886 +0800
+++ workdir/deno_core-0.200.0/Cargo.toml    2023-08-21 18:15:47.362783080 +0800
@@ -91,7 +91,7 @@
 ]

 [dependencies.v8]
-version = "0.74.3"
+version = "0.75.0"
 default-features = false

 [dev-dependencies.bencher]
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_core-0.200.0/runtime/jsruntime.rs workdir/deno_core-0.200.0/runtime/jsruntime.rs
--- workdir-1/deno_core-0.200.0/runtime/jsruntime.rs    2023-08-21 14:23:37.216217840 +0800
+++ workdir/deno_core-0.200.0/runtime/jsruntime.rs  2023-08-21 18:15:47.365783210 +0800
@@ -326,7 +326,7 @@
   #[repr(C, align(16))]
   struct IcuData([u8; 10541264]);
   static ICU_DATA: IcuData = IcuData(*include_bytes!("icudtl.dat"));
-  v8::icu::set_common_data_72(&ICU_DATA.0).unwrap();
+  v8::icu::set_common_data_73(&ICU_DATA.0).unwrap();

   let base_flags = concat!(
     " --wasm-test-streaming",
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_fs-0.25.0/std_fs.rs workdir/deno_fs-0.25.0/std_fs.rs
--- workdir-1/deno_fs-0.25.0/std_fs.rs  2023-08-21 15:46:24.398685329 +0800
+++ workdir/deno_fs-0.25.0/std_fs.rs    2023-08-21 18:15:47.367783296 +0800
@@ -62,7 +62,7 @@
       let _ = umask(prev);
       prev
     };
-    #[cfg(target_os = "linux")]
+    #[cfg(any(target_os = "linux", target_os = "android"))]
     {
       Ok(r.bits())
     }
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_node-0.52.0/ops/os.rs workdir/deno_node-0.52.0/ops/os.rs
--- workdir-1/deno_node-0.52.0/ops/os.rs    2023-08-21 15:50:56.641553194 +0800
+++ workdir/deno_node-0.52.0/ops/os.rs  2023-08-21 18:15:47.388784205 +0800
@@ -60,7 +60,7 @@
   use libc::id_t;
   use libc::PRIO_PROCESS;

-  #[cfg(target_os = "macos")]
+  #[cfg(any(target_os = "macos", target_os = "android"))]
   #[allow(non_camel_case_types)]
   type priority_t = i32;
   #[cfg(target_os = "linux")]
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_node-0.52.0/polyfills/internal_binding/uv.ts workdir/deno_node-0.52.0/polyfills/internal_binding/uv.ts
--- workdir-1/deno_node-0.52.0/polyfills/internal_binding/uv.ts 2023-08-21 15:50:56.701556428 +0800
+++ workdir/deno_node-0.52.0/polyfills/internal_binding/uv.ts   2023-08-21 20:06:42.203785426 +0800
@@ -494,6 +494,8 @@
     ? codeToErrorDarwin
     : osType === "linux"
     ? codeToErrorLinux
+    : osType === "android"
+    ? codeToErrorLinux
     : osType === "freebsd"
     ? codeToErrorFreebsd
     : osType === "openbsd"
@@ -508,6 +510,8 @@
     ? errorToCodeDarwin
     : osType === "linux"
     ? errorToCodeLinux
+    : osType === "android"
+    ? errorToCodeLinux
     : osType === "freebsd"
     ? errorToCodeFreebsd
     : osType === "openbsd"
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_node-0.52.0/polyfills/_util/os.ts workdir/deno_node-0.52.0/polyfills/_util/os.ts
--- workdir-1/deno_node-0.52.0/polyfills/_util/os.ts    2023-08-21 15:50:56.670554757 +0800
+++ workdir/deno_node-0.52.0/polyfills/_util/os.ts  2023-08-21 20:07:47.396796206 +0800
@@ -2,7 +2,7 @@

 const { ops } = globalThis.__bootstrap.core;

-export type OSType = "windows" | "linux" | "darwin" | "freebsd" | "openbsd";
+export type OSType = "windows" | "linux" | "android" | "darwin" | "freebsd" | "openbsd";

 export const osType: OSType = ops.op_node_build_os();

diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/build.rs workdir/deno_runtime-0.123.0/build.rs
--- workdir-1/deno_runtime-0.123.0/build.rs 2023-08-21 13:44:12.370122735 +0800
+++ workdir/deno_runtime-0.123.0/build.rs   2023-08-21 18:15:47.391784335 +0800
@@ -107,15 +107,6 @@
     }
   }

-  impl deno_ffi::FfiPermissions for Permissions {
-    fn check_partial(
-      &mut self,
-      _path: Option<&Path>,
-    ) -> Result<(), deno_core::error::AnyError> {
-      unreachable!("snapshotting!")
-    }
-  }
-
   impl deno_napi::NapiPermissions for Permissions {
     fn check(
       &mut self,
@@ -257,7 +248,6 @@
       deno_broadcast_channel,
       // FIXME(bartlomieju): this should be reenabled
       // "deno_node",
-      deno_ffi,
       deno_net,
       deno_napi,
       deno_http,
@@ -333,7 +323,6 @@
         deno_broadcast_channel::InMemoryBroadcastChannel::default(),
         false, // No --unstable.
       ),
-      deno_ffi::deno_ffi::init_ops_and_esm::<Permissions>(false),
       deno_net::deno_net::init_ops_and_esm::<Permissions>(
         None, false, // No --unstable.
         None,
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/Cargo.toml workdir/deno_runtime-0.123.0/Cargo.toml
--- workdir-1/deno_runtime-0.123.0/Cargo.toml   2023-08-21 13:44:12.369122687 +0800
+++ workdir/deno_runtime-0.123.0/Cargo.toml 2023-08-21 18:15:47.392784379 +0800
@@ -62,9 +62,6 @@
 [dependencies.deno_fetch]
 version = "0.139.0"

-[dependencies.deno_ffi]
-version = "0.102.0"
-
 [dependencies.deno_fs]
 version = "0.25.0"
 features = ["sync_fs"]
@@ -202,9 +199,6 @@
 [build-dependencies.deno_fetch]
 version = "0.139.0"

-[build-dependencies.deno_ffi]
-version = "0.102.0"
-
 [build-dependencies.deno_fs]
 version = "0.25.0"
 features = ["sync_fs"]
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/js/10_permissions.js workdir/deno_runtime-0.123.0/js/10_permissions.js
--- workdir-1/deno_runtime-0.123.0/js/10_permissions.js 2023-08-21 13:44:12.374122924 +0800
+++ workdir/deno_runtime-0.123.0/js/10_permissions.js   2023-08-21 18:15:47.393784422 +0800
@@ -33,7 +33,7 @@
  * @property {boolean} partial
  */

-/** @type {ReadonlyArray<"read" | "write" | "net" | "env" | "sys" | "run" | "ffi" | "hrtime">} */
+/** @type {ReadonlyArray<"read" | "write" | "net" | "env" | "sys" | "run" | "hrtime">} */
 const permissionNames = [
   "read",
   "write",
@@ -41,7 +41,6 @@
   "env",
   "sys",
   "run",
-  "ffi",
   "hrtime",
 ];

@@ -129,7 +128,7 @@
 function cache(desc, rawStatus) {
   let { name: key } = desc;
   if (
-    (desc.name === "read" || desc.name === "write" || desc.name === "ffi") &&
+    (desc.name === "read" || desc.name === "write") &&
     ReflectHas(desc, "path")
   ) {
     key += `-${desc.path}&`;
@@ -180,7 +179,7 @@
  */
 function formDescriptor(desc) {
   if (
-    desc.name === "read" || desc.name === "write" || desc.name === "ffi"
+    desc.name === "read" || desc.name === "write"
   ) {
     desc.path = pathFromURL(desc.path);
   } else if (desc.name === "run") {
@@ -266,7 +265,7 @@
   if (typeof permissions == "object" && permissions != null) {
     const serializedPermissions = {};
     for (
-      const key of new SafeArrayIterator(["read", "write", "run", "ffi"])
+      const key of new SafeArrayIterator(["read", "write", "run"])
     ) {
       if (ArrayIsArray(permissions[key])) {
         serializedPermissions[key] = ArrayPrototypeMap(
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/js/90_deno_ns.js workdir/deno_runtime-0.123.0/js/90_deno_ns.js
--- workdir-1/deno_runtime-0.123.0/js/90_deno_ns.js 2023-08-21 13:44:12.377123065 +0800
+++ workdir/deno_runtime-0.123.0/js/90_deno_ns.js   2023-08-21 18:15:47.414785331 +0800
@@ -5,7 +5,6 @@
 import * as timers from "ext:deno_web/02_timers.js";
 import * as httpClient from "ext:deno_fetch/22_http_client.js";
 import * as console from "ext:deno_console/01_console.js";
-import * as ffi from "ext:deno_ffi/00_ffi.js";
 import * as net from "ext:deno_net/01_net.js";
 import * as tls from "ext:deno_net/02_tls.js";
 import * as http from "ext:deno_http/01_http.js";
@@ -163,11 +162,6 @@
   createHttpClient: httpClient.createHttpClient,
   // TODO(bartlomieju): why is it needed?
   http,
-  dlopen: ffi.dlopen,
-  UnsafeCallback: ffi.UnsafeCallback,
-  UnsafePointer: ffi.UnsafePointer,
-  UnsafePointerView: ffi.UnsafePointerView,
-  UnsafeFnPointer: ffi.UnsafeFnPointer,
   flock: fs.flock,
   flockSync: fs.flockSync,
   funlock: fs.funlock,
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/lib.rs workdir/deno_runtime-0.123.0/lib.rs
--- workdir-1/deno_runtime-0.123.0/lib.rs   2023-08-21 13:44:12.378123113 +0800
+++ workdir/deno_runtime-0.123.0/lib.rs 2023-08-21 18:15:47.417785461 +0800
@@ -6,7 +6,6 @@
 pub use deno_core;
 pub use deno_crypto;
 pub use deno_fetch;
-pub use deno_ffi;
 pub use deno_fs;
 pub use deno_http;
 pub use deno_io;
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/ops/os/mod.rs workdir/deno_runtime-0.123.0/ops/os/mod.rs
--- workdir-1/deno_runtime-0.123.0/ops/os/mod.rs    2023-08-21 13:44:12.380123207 +0800
+++ workdir/deno_runtime-0.123.0/ops/os/mod.rs  2023-08-21 18:15:47.419785548 +0800
@@ -305,7 +305,7 @@
   }
 }

-#[cfg(target_os = "linux")]
+#[cfg(any(target_os = "linux", target_os = "android"))]
 fn rss() -> usize {
   // Inspired by https://github.com/Arc-blroth/memory-stats/blob/5364d0d09143de2a470d33161b2330914228fde9/src/linux.rs

diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/ops/os/sys_info.rs workdir/deno_runtime-0.123.0/ops/os/sys_info.rs
--- workdir-1/deno_runtime-0.123.0/ops/os/sys_info.rs   2023-08-21 13:44:12.380123207 +0800
+++ workdir/deno_runtime-0.123.0/ops/os/sys_info.rs 2023-08-21 18:15:47.423785721 +0800
@@ -6,7 +6,7 @@
 const DEFAULT_LOADAVG: LoadAvg = (0.0, 0.0, 0.0);

 pub fn loadavg() -> LoadAvg {
-  #[cfg(target_os = "linux")]
+  #[cfg(any(target_os = "linux", target_os = "android"))]
   {
     use libc::SI_LOAD_SHIFT;

@@ -117,6 +117,10 @@
       )
     }
   }
+  #[cfg(target_os = "android")]
+  {
+    String::from("")
+  }
 }

 #[cfg(target_family = "windows")]
@@ -198,7 +202,7 @@
     swap_total: 0,
     swap_free: 0,
   };
-  #[cfg(target_os = "linux")]
+  #[cfg(any(target_os = "linux", target_os = "android"))]
   {
     let mut info = std::mem::MaybeUninit::uninit();
     // SAFETY: `info` is a valid pointer to a `libc::sysinfo` struct.
@@ -331,7 +335,7 @@
 pub fn os_uptime() -> u64 {
   let uptime: u64;

-  #[cfg(target_os = "linux")]
+  #[cfg(any(target_os = "linux", target_os = "android"))]
   {
     let mut info = std::mem::MaybeUninit::uninit();
     // SAFETY: `info` is a valid pointer to a `libc::sysinfo` struct.
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/permissions/mod.rs workdir/deno_runtime-0.123.0/permissions/mod.rs
--- workdir-1/deno_runtime-0.123.0/permissions/mod.rs   2023-08-21 13:44:12.381123255 +0800
+++ workdir/deno_runtime-0.123.0/permissions/mod.rs 2023-08-21 18:15:47.429785981 +0800
@@ -1464,13 +1464,6 @@
   }
 }

-impl deno_ffi::FfiPermissions for PermissionsContainer {
-  #[inline(always)]
-  fn check_partial(&mut self, path: Option<&Path>) -> Result<(), AnyError> {
-    self.0.lock().ffi.check_partial(path)
-  }
-}
-
 impl deno_kv::sqlite::SqliteDbHandlerPermissions for PermissionsContainer {
   #[inline(always)]
   fn check_read(&mut self, p: &Path, api_name: &str) -> Result<(), AnyError> {
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/web_worker.rs workdir/deno_runtime-0.123.0/web_worker.rs
--- workdir-1/deno_runtime-0.123.0/web_worker.rs    2023-08-21 13:44:12.382123302 +0800
+++ workdir/deno_runtime-0.123.0/web_worker.rs  2023-08-21 18:15:47.435786241 +0800
@@ -433,7 +433,6 @@
         options.broadcast_channel.clone(),
         unstable,
       ),
-      deno_ffi::deno_ffi::init_ops_and_esm::<PermissionsContainer>(unstable),
       deno_net::deno_net::init_ops_and_esm::<PermissionsContainer>(
         options.root_cert_store_provider.clone(),
         unstable,
diff '--color=auto' -ru -x Cargo.lock workdir-1/deno_runtime-0.123.0/worker.rs workdir/deno_runtime-0.123.0/worker.rs
--- workdir-1/deno_runtime-0.123.0/worker.rs    2023-08-21 13:44:12.382123302 +0800
+++ workdir/deno_runtime-0.123.0/worker.rs  2023-08-21 18:15:47.438786370 +0800
@@ -334,7 +334,6 @@
         options.broadcast_channel.clone(),
         unstable,
       ),
-      deno_ffi::deno_ffi::init_ops_and_esm::<PermissionsContainer>(unstable),
       deno_net::deno_net::init_ops_and_esm::<PermissionsContainer>(
         options.root_cert_store_provider.clone(),
         unstable,
diff '--color=auto' -ru -x Cargo.lock workdir-1/libz-ng-sys-1.1.12/build_zng.rs workdir/libz-ng-sys-1.1.12/build_zng.rs
--- workdir-1/libz-ng-sys-1.1.12/build_zng.rs   2023-08-21 15:28:56.175241480 +0800
+++ workdir/libz-ng-sys-1.1.12/build_zng.rs 2023-08-21 18:15:47.439786414 +0800
@@ -17,6 +17,10 @@
     if target == "i686-pc-windows-msvc" {
         cmake.define("CMAKE_GENERATOR_PLATFORM", "Win32");
     }
+    // Android
+    if target.contains("android") {
+        cmake.define("CMAKE_ANDROID_NDK", env::var("ANDROID_NDK_HOME").unwrap());
+    }

     let install_dir = cmake.build();

diff '--color=auto' -ru -x Cargo.lock workdir-1/serde_v8-0.111.0/Cargo.toml workdir/serde_v8-0.111.0/Cargo.toml
--- workdir-1/serde_v8-0.111.0/Cargo.toml   2023-08-21 14:25:49.665241353 +0800
+++ workdir/serde_v8-0.111.0/Cargo.toml 2023-08-21 18:15:47.440786457 +0800
@@ -58,7 +58,7 @@
 version = "1.0.40"

 [dependencies.v8]
-version = "0.74.3"
+version = "0.75.0"
 default-features = false

 [dev-dependencies.bencher]
secext2022 commented 1 year ago

The dependency libz-ng-sys 1.1.12 of deno 1.36.1 need patch to build for aarch64-linux-android.

https://github.com/rust-lang/libz-sys/issues/149

secext2022 commented 1 year ago

Build deno for x86_64-linux-android is also ok, with a little more patch.

(https://github.com/fm-elpac/v8-src/issues/1)

secext2022 commented 6 months ago

The new version of deno can not run on Android, because deno_core crash.

https://github.com/denoland/deno_core/issues/738

CodeIter commented 6 months ago

I made a shell script termux-pacman-glibc-setup.sh to setup glibc-runner with pacman on Termux and install Deno.JS and Bun.JS as a demo.

secext2022 commented 6 months ago

@CodeIter interesting !

I tries your method, and it seems that the offical aarch64-unknown-linux-gnu deno build can just run on Android (termux).

We just need to use patchelf and provide some library files (*.so), like this:

$ patchelf --set-rpath /data/data/com.termux/files/usr/glibc/lib --set-interpreter /data/data/com.termux/files/usr/glibc/lib/ld-linux-aarch64.so.1 deno
$ unset LD_PRELOAD
$ ./deno --version
deno 1.43.3 (release, aarch64-unknown-linux-gnu)
v8 12.4.254.13
typescript 5.4.5
$ ./deno
Deno 1.43.3
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> 0.1+0.2
0.30000000000000004
>
secext2022 commented 6 months ago
$ ldd deno
        libdl.so.2 => /data/data/com.termux/files/usr/glibc/lib/libdl.so.2
        libgcc_s.so.1 => /data/data/com.termux/files/usr/glibc/lib/libgcc_s.so.1
        libpthread.so.0 => /data/data/com.termux/files/usr/glibc/lib/libpthread.so.0
        libm.so.6 => /data/data/com.termux/files/usr/glibc/lib/libm.so.6
        libc.so.6 => /data/data/com.termux/files/usr/glibc/lib/libc.so.6
        ld-linux-aarch64.so.1 => /data/data/com.termux/files/usr/glibc/lib/ld-linux-aarch64.so.1
$
secext2022 commented 6 months ago

This method can also work out of termux: (adb shell)

raphael:/data/local/tmp $ pwd
/data/local/tmp
raphael:/data/local/tmp $ ls -l 
total 138648
-rwxrwxrwx 1 shell shell 141959425 2024-05-17 06:57 deno
drwxrwxr-x 2 shell shell      4096 2024-05-17 06:54 lib
raphael:/data/local/tmp $ ls -l lib
total 4240
-rwxrwxrwx 1 shell shell  241064 2024-05-17 06:53 ld-linux-aarch64.so.1
-rwxrwxrwx 1 shell shell 2292352 2024-05-17 06:53 libc.so.6
-rwxrwxrwx 1 shell shell   69736 2024-05-17 06:53 libdl.so.2
-rw-rw-rw- 1 shell shell  591400 2024-05-17 06:53 libgcc_s.so.1
-rwxrwxrwx 1 shell shell 1039216 2024-05-17 06:53 libm.so.6
-rwxrwxrwx 1 shell shell   70120 2024-05-17 06:53 libpthread.so.0
raphael:/data/local/tmp $ export HOME=$(pwd)                                                                                  
raphael:/data/local/tmp $ ./deno --version
deno 1.43.3 (release, aarch64-unknown-linux-gnu)
v8 12.4.254.13
typescript 5.4.5
raphael:/data/local/tmp $ ./deno
Deno 1.43.3
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> 0.1 + 0.2
0.30000000000000004
> Deno.version
{ deno: "1.43.3", v8: "12.4.254.13", typescript: "5.4.5" }
> 

patch deno on host:

> patchelf --set-rpath /data/local/tmp/lib --set-interpreter /data/local/tmp/lib/ld-linux-aarch64.so.1 deno

so, this is a really simple and great method ! Thank you !

CodeIter commented 6 months ago

I have updated the script.

SerJaimeLannister commented 2 weeks ago

https://nodejs-mobile.github.io/ is there something like this equivalent on deno world (ie. instead of having qemu , we can have something like this)

another note is that you should probably look at blink (https://github.com/jart/blink) , its faster than qemu (but its for x86_64) so I don't really know. But I guess the main point is that it can run specific applications instead of a whole system like qemu (I may be wrong on this part because I haven't dabbled in qemu)