denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97.35k stars 5.36k forks source link

Deno build on riscv64 #18702

Open hack3ric opened 1 year ago

hack3ric commented 1 year ago

I've successfully built Deno (1.32.3, 1.32.4 in progress) in qemu-user and run on HiFive Unmatched on Arch Linux riscv64.

Some tweaks I did:

Environment variables exported:

local _extra_gn_args=(
  'custom_toolchain="//build/toolchain/linux/unbundle:default"'
  'host_toolchain="//build/toolchain/linux/unbundle:default"'
  'v8_enable_shared_ro_heap=true'
)

export CC=clang CXX=clang++ AR=ar NM=nm
export CFLAGS="${CFLAGS//-fstack-clash-protection/}" CXXFLAGS="${CXXFLAGS//-fstack-clash-protection/}"
export V8_FROM_SOURCE=1
export CLANG_BASE_PATH=/usr
export GN=/usr/bin/gn NINJA=/usr/bin/ninja
export EXTRA_GN_ARGS="${_extra_gn_args[@]}"
export NO_PRINT_GN_ARGS=1

Build script for Arch Linux can be found in https://github.com/felixonmars/archriscv-packages/pull/2510. Patch Arch Linux's official PKGBUILD and build it:

pacman -S asp
asp checkout deno
cd deno/repos/community-x86_64
curl -L https://github.com/felixonmars/archriscv-packages/pull/2510.patch | git apply
makepkg -s

Caveats

Note that current build runs TypeScript scripts normally, but in REPL errors will appear (while not affecting basic auto completion functionality):

ERROR TSLS = Error: Could not find source file: 'file://<working_dir>/$deno$repl.ts'.
    at getValidSourceFile (ext:deno_tsc/00_typescript.js:133358:23)
    at Object.getSemanticDiagnostics (ext:deno_tsc/00_typescript.js:133600:32)
    at serverRequest (ext:deno_tsc/99_main_compiler.js:1059:34)
    at [ext:cli/lsp/tsc.rs:3496:26]:1:12

This is triggered only by TypeScript LSP's getDiagnostics request. Other LSP requests with specifiers containing $deno$repl.ts seem to not fail. I'm yet to find out why.

kxxt commented 1 year ago

The errors from TypeScript LSP is perhaps caused by some path manipulation operations in 00_typescript.js.

In function pathForLibFile in 00_typescript.js:

function hexEncode(s) {
  // helper function for debugging
  let result = "";
  for(let i=0; i < s.length; i++) {
    let hex = s.charCodeAt(i).toString(16);
    result += ("000" + hex).slice(-4);
  }
  return result;
}
function pathForLibFile(libFileName) {
  const core = globalThis.Deno.core;
  core.print(`entering pathForLibFile(${libFileName}): \n`);
  const components = libFileName.split(".");
  let path = components[1];
  core.print(`!!!!! path = ${path}, components = ${components}\n`, true);
  let i = 2;
  while (components[i] && components[i] !== "d") {
    let expr = (i === 2 ? "/" : "-") + components[i];
    core.print(`!!!!! path += ${hexEncode(expr)}\n`, true);
    path += expr;
    i++;
  }
  const resolveFrom = combinePaths(currentDirectory, `__lib_node_modules_lookup_${libFileName}__.ts`);
  core.print(`pathForLibFile(${libFileName}): \n    defaultLibraryPath = ${defaultLibraryPath}\n    resolveFrom = ${resolveFrom}\n    path = ${path}\n`, true);
  const localOverrideModuleResult = resolveModuleName("@typescript/lib-" + path, resolveFrom, { moduleResolution: 2 /* Node10 */ }, host, moduleResolutionCache);
  if (localOverrideModuleResult == null ? void 0 : localOverrideModuleResult.resolvedModule) {
    return localOverrideModuleResult.resolvedModule.resolvedFileName;
  }
  return combinePaths(defaultLibraryPath, libFileName);
}

The while loop is producing wrong results on riscv64:

before call processRootFile: lib.deno.ns.d.ts   entering pathForLibFile(lib.deno.ns.d.ts):
!!!!! path = deno, components = lib,deno,ns,d,ts        !!!!! path += 0060006e0073      !!!!! path += 000b0064  !!!!! path += 000b00740073      pathForLibFile(lib.deno.ns.d.ts):
defaultLibraryPath = asset:///lib.esnext.d.ts
resolveFrom = cache:///__lib_node_modules_lookup_lib.deno.ns.d.ts__.ts
path = deno`ns
d
ts   resolveModuleName(@typescript/lib-deno`ns
d
ts, cache:///__lib_node_modules_lookup_lib.deno.ns.d.ts__.ts, [object Object], [object Object], undefined, undefined, undefined)

image

The log doesn't look nice. There are some extra characters:

for "ns", there is an extra U+0060(`) before it
for "d" and "ts", there is an extra U+000B(Line Tabulation) before it
(The loop actually shouldn't reach here, the correct result is "deno/ns")

It looks like a V8 bug.

By explicitly using char codes I can workaround this bug:

// while (components[i] && components[i] !== "d") {
while (components[i] && components[i].charCodeAt(0) !== 100) {
// let expr = (i === 2 ? "/" : "-") + components[i];
  let expr;
  if (i === 2)
        expr = String.fromCharCode(47) + components[i];
  else
        expr = String.fromCharCode(45) + components[i];

But there are other places that have bugs about path manipulation operations. So only fixing this one place can't let that error go away.

Any suggestions and helps will be well appreciated!

hack3ric commented 1 year ago

Hello, I want to update the status of Deno on riscv64.

On 1.33.1 https://github.com/denoland/rusty_v8/pull/1209 is merged and released. The $deno$repl.ts problem disappeared after disabling v8_enable_shared_ro_heap (as GN parameter restriction was removed in https://github.com/denoland/rusty_v8/commit/07f2e9f3b63ada1a4cd3205a5b13f2ba25cd2ca6). Other parameters are unchanged.

local _extra_gn_args=(
  'custom_toolchain="//build/toolchain/linux/unbundle:default"'
  'host_toolchain="//build/toolchain/linux/unbundle:default"'
)

export CC=clang CXX=clang++ AR=ar NM=nm
export CFLAGS="${CFLAGS//-fstack-clash-protection/}" CXXFLAGS="${CXXFLAGS//-fstack-clash-protection/}"
export V8_FROM_SOURCE=1
export CLANG_BASE_PATH=/usr
export GN=/usr/bin/gn NINJA=/usr/bin/ninja
export EXTRA_GN_ARGS="${_extra_gn_args[@]}"
export NO_PRINT_GN_ARGS=1

As a side note, V8's pointer compression on riscv64 is fixed in https://chromium-review.googlesource.com/c/v8/v8/+/4559353. If the aforementioned restriction is added back, Deno is likely to still work properly.

tuler commented 3 weeks ago

What is the status of deno on riscv64 now that deno v2 is out? Any effort in progress?

hack3ric commented 3 weeks ago

What is the status of deno on riscv64 now that deno v2 is out? Any effort in progress?

Pretty much usable AFAIK, at least on recent versions <2. 1.46.0 needs to disable pointer compression like before. Progress are tracked in Arch Linux RISC-V's deno patch against Arch Linux PKGBUILD (if this file does not exist then the original PKGBUILD will build fine.)

As for Deno 2.0, I haven't looked at it yet, but as long as it doesn't have groundbreaking changes in (rusty-)V8, it should build and work with no to little effort of porting.

shodan8192 commented 1 week ago

Hello, I'm trying deno-1.46.3-2-riscv64 from archriscv.felixc.at on LicheeRV Nano board with @Fishwaldo's Debian.

On some scripts I get Illegal instruction, for example it can be triggered by :

echo 'import mqtt from "npm:mqtt"' | deno run -

Through gdb I've found that it's caused by vle8.v which I believe is RVV 1.0 instruction. Is my example giving the same error on HiFive Unmatched ?

hack3ric commented 1 week ago

Hi @shodan8192 My guess is that your kernel/SBI on Lichee Pi Nano treats RVV 0.7 in C906 cores as RVV 1.0. Could you try upgrading OpenSBI and see if it works?

shodan8192 commented 1 week ago

My system report rv64imafdvcsu ISA . SiFive U740 doesn't have any vector extension, that's why I've asked if my example works on it. If yes - then I have to disable RVV in kernel, if not, then deno should be compiled targeting cpu without vector extension.

hack3ric commented 1 week ago

This is not statically compiled code, it's generated by V8.

In terms of your question -- yes, it runs fine on Unmatched, and SG2042 (which has its RVV 0.7 hidden so application won't recognize it as RVV anymore)

shodan8192 commented 1 week ago

Now that's clear. I didn't knew that V8 make use of vector instructions if they're available, thats why I though they were generated by compiler. So I'll try to disable RVV 0.7 and make things work. Thanks @hack3ric !

shodan8192 commented 1 week ago

Sorry for spam, but I can't get it to work on Lichee Pi Nano with 5.10 kernel.

I've removed CONFIG_VECTOR and CONFIG_VECTOR_0_7 from kernel config and set riscv,isa = "rv64imafdc" in dts file then rebuild - and nothing changed. Testing with this code :

#include <sys/auxv.h>
#include <stdio.h>
#include <asm/hwcap.h>

void main() {
  unsigned long hw_cap = getauxval(AT_HWCAP);
  printf("I %s\n", hw_cap & COMPAT_HWCAP_ISA_I ? "detected" : "not found");
  printf("M %s\n", hw_cap & COMPAT_HWCAP_ISA_M ? "detected" : "not found");
  printf("A %s\n", hw_cap & COMPAT_HWCAP_ISA_A ? "detected" : "not found");
  printf("F %s\n", hw_cap & COMPAT_HWCAP_ISA_F ? "detected" : "not found");
  printf("D %s\n", hw_cap & COMPAT_HWCAP_ISA_D ? "detected" : "not found");
  printf("C %s\n", hw_cap & COMPAT_HWCAP_ISA_C ? "detected" : "not found");
  printf("V %s\n", hw_cap & COMPAT_HWCAP_ISA_V ? "detected" : "not found");
}

shows :

I detected
M detected
A detected
F detected
D detected
C detected
V not found

/proc/cpuinfo :

processor   : 0
hart        : 0
isa     : rv64imafdc
mmu     : sv39

Running my example with V8 JIT disabled :

echo 'import mqtt from "npm:mqtt"' | deno run --v8-flags=--jitless -

also crashes with SIGILL, so it seems it's not in V8 generated code. What else I can try to see what happening ?

hack3ric commented 1 week ago

It could be another problem with C906. fence.tso was originally not included in G, and C906 doesn't implement it in hardware. Newer OpenSBI for LicheeRV Nano (and other boards that uses C906 cores) should emulate it. Could you try upgrading that?

p.s. Sorry for not replying the email, since you also asked here later I'll answer here.

shodan8192 commented 1 week ago

I'm using OpenSBI already patched for fence.tso emulation, and illegal instruction is still one of RVV.

hack3ric commented 1 week ago

I still cannot reproduce this on LicheeRV Nano running Arch Linux RISC-V (kernel 6.11.3-arch1-1), so I'm out of ideas...

shodan8192 commented 1 week ago

Kernel 6.11 have riscv_hwprobe syscall. If it's used to get cpu capabilities, and kernel 5.11 doesn't have it, then it can be erroneously assumed that cpu has RVV.

Where I can find image of Arch Linux you using ?

hack3ric commented 1 week ago

I'm using one of our team's build machine. There's no image right now, but I assume you can flash Debian first, replace the chroot, and then modify extlinux.conf or something so it boots to Arch-managed kernel.

Deno on Arch RISC-V is built against newer version of glibc (2.40 actually), and it may not work under your Debian image. As you mentioned in the email the detection method used in V8 is compile-time.

$ echo '#include <stdio.h>' | gcc -dM -E - | grep GLIBC
#define __GLIBC_USE(F) __GLIBC_USE_ ## F
#define __GLIBC_PREREQ(maj,min) ((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
#define __GLIBC_USE_IEC_60559_TYPES_EXT 0
#define __GLIBC__ 2
#define __GLIBC_USE_IEC_60559_BFP_EXT 0
#define __GLIBC_USE_DEPRECATED_GETS 0
#define __GLIBC_USE_C23_STRTOL 0
#define __GLIBC_USE_DEPRECATED_SCANF 0
#define __GLIBC_USE_LIB_EXT2 0
#define __GLIBC_USE_IEC_60559_FUNCS_EXT 0
#define __GLIBC_USE_IEC_60559_FUNCS_EXT_C23 0
#define __GLIBC_USE_IEC_60559_BFP_EXT_C23 0
#define __GLIBC_USE_ISOC23 0
#define __GLIBC_MINOR__ 40
#define __GLIBC_USE_IEC_60559_EXT 0

You could try to compile and install a newer version of glibc (>=2.40) on Debian though. But yeah, use other distros' packages is considered a bad idea :(

shodan8192 commented 1 week ago

In my Debian image there's the same version of glibc ldd (Debian GLIBC 2.40-3) 2.40 as in your Arch. I think that deno is quite distro-agnostic and should run on any Linux with recent glibc version.

I've excluded V8 as crash cause by disabling JIT, and by src code analysing - in my case V8 doesn't try use of rvv nor fpu, because detection method doesn't cover my case (glibc 2.40 and old kernel without hwprobe)

Actually hwprobe syscall is called triple times before crash and only last of them originating from V8 :

syscall_0x102(0x3fffbf6570, 0x1, 0, 0, 0, 0) = -1 ENOSYS (Function not implemented)
syscall_0x102(0x3fffbf6570, 0x1, 0, 0, 0, 0) = -1 ENOSYS (Function not implemented)
syscall_0x102(0x3fffbeded8, 0x1, 0, 0, 0, 0x2ab59305d0) = -1 ENOSYS (Function not implemented)

I'll investigate it further.

shodan8192 commented 3 days ago

OK, so I found the problem. In my example crash occur, when npm tgz package is downloaded and is about to decompress. Deno is using flate2 library for that : https://github.com/denoland/deno/blob/2c8a0e791732ab00907ca11c3a4918ad26ead03f/cli/npm/managed/cache/tarball_extract.rs#L17 with zlib-ng C library as the backend, which currently has this issue : https://github.com/zlib-ng/zlib-ng/issues/1705 due to RVV detection method.

On my system, if I spoof kernel version to6.10.4 deno works without crash.