[Bug]: 4.0.2 armhf and arm64 tarballs error out on `@parcel/watcher`

aptalca commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

OS/Web Information

Web Browser: Chrome
Local OS: Win10
Remote OS: Ubuntu
Remote Architecture: aarch64
code-server --version: 4.0.2

Steps to Reproduce

Download and extract linux arm64 tarball for 4.0.2
Fix the argon2 issue (specific to 4.0.2) by installing the module
Start code-server

Expected

No errors

Actual

Error about @parcel/watcher

Logs

[04:31:53] Extension host agent started.
[04:31:55] [192.168.13.58][2ae76c94][ManagementConnection] New connection established.
[04:31:57] [192.168.13.58][ae08c7ad][ExtensionHostConnection] New connection established.
[04:31:57] [192.168.13.58][ae08c7ad][ExtensionHostConnection] <1551> Launched Extension Host Process.
[IPC Library: File Watcher (parcel, node.js)] Loading "@parcel/watcher" failed
[IPC Library: File Watcher (parcel, node.js)] Error: No native build was found for platform=linux arch=arm64 runtime=node abi=83 uv=1 armv=8 libc=glibc node=14.17.4
    loaded from: /app/code-server/vendor/modules/code-oss-dev/node_modules/@parcel/watcher

    at Function.load.path (/app/code-server/vendor/modules/code-oss-dev/node_modules/node-gyp-build/index.js:59:9)
    at load (/app/code-server/vendor/modules/code-oss-dev/node_modules/node-gyp-build/index.js:21:30)
    at Object.<anonymous> (/app/code-server/vendor/modules/code-oss-dev/node_modules/@parcel/watcher/index.js:1:42)
    at Module._compile (internal/modules/cjs/loader.js:1072:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
    at Module.load (internal/modules/cjs/loader.js:937:32)
    at Function.Module._load (internal/modules/cjs/loader.js:778:12)
    at Module.require (internal/modules/cjs/loader.js:961:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at r (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:101)
    at e.load (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:4:1723)
    at e.load (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:3:10695)
    at d (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:10300)
    at Object.errorback (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:10421)
    at e.triggerErrorback (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:3:11059)
    at /app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:3:10765
    at e.load (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:4:1740)
    at e.load (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:3:10695)
    at d (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:10300)
    at i._loadModule (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:10430)
    at i._resolve (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:6:452)
    at i.defineModule (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:5:6142)
    at v (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:6:1702)
    at g (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:4:2557)
    at Object.<anonymous> (/app/code-server/vendor/modules/code-oss-dev/out/vs/platform/files/node/watcher/parcel/watcherApp.js:12:22221)
    at Object.<anonymous> (/app/code-server/vendor/modules/code-oss-dev/out/vs/platform/files/node/watcher/parcel/watcherApp.js:12:30915)
    at e._createAndEvalScript (/app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:4:2601)
    at /app/code-server/vendor/modules/code-oss-dev/out/vs/loader.js:4:2240
    at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:73:3)
[IPC Library: File Watcher (parcel, node.js)] Here are the modules that depend on it:
[IPC Library: File Watcher (parcel, node.js)]% [ 'vs/platform/files/node/watcher/parcel/parcelWatcherService' ]

Screenshot/Video

No response

Does this issue happen in VS Code?

[X] I cannot reproduce this in VS Code.

Are you accessing code-server over HTTPS?

[X] I am using HTTPS.

Notes

No response

aptalca commented 2 years ago

By the way, I saw the issue reports about @node-rs/argon2 and bugs with npm and cross-compiling (likely this issue is related as the watcher module works just fine on amd64, as well as when code-server is installed via yarn on arm). Did you consider using docker/qemu to build the armhf and aarch64 tarballs so you don't have to worry about such arch related bugs?

jsjoeio commented 2 years ago

This is interesting...So I opened a bug report semi-related to this back in January: https://github.com/parcel-bundler/watcher/issues/93

But we never arrived at a solution.

I think the build from source fix we added back when reverting the argon2 change should fix this 🤔 https://github.com/coder/code-server/pull/4829

Did you consider using docker/qemu to build the armhf and aarch64 tarballs so you don't have to worry about such arch related bugs?

When we were debugging, we did try using qemu locally on my macOS to emulate an aarch64 env but @code-asher and I ran into a ton of issues.

I think we are qemu in our docker workflow here:

https://github.com/coder/code-server/blob/8135d2ecc3a4cd8b4d42989ee32c4d6b745d4d20/.github/workflows/docker.yaml#L19-L20

But I guess that would only affect the Docker builds.

I wonder if we should be using it in our main workflow here:

https://github.com/coder/code-server/blob/8135d2ecc3a4cd8b4d42989ee32c4d6b745d4d20/.github/workflows/ci.yaml#L164-L168

I don't know if that would break other stuff though. We'd have to look into it more.

aptalca commented 2 years ago

Right, qemu has some bugs with runtime, but builds are much more reliable.

Assuming the release package generated via yarn release (npm-package) indeed does not contain any binaries or os/arch specific things as the note says, then I'm thinking this part could be using qemu instead of cross-compile: https://github.com/coder/code-server/blob/8135d2ecc3a4cd8b4d42989ee32c4d6b745d4d20/.github/workflows/ci.yaml#L285-L296 and that would also solve the argon2 issue. The whole standalone package creation step (including for amd64) could be a 3 option matrix, one for each arch.

And just to give you a heads up (not sure if this will be an issue for the next release or not), we, at linuxserver.io, publish docker images for code-server on amd64, armhf and aarch64 and we also do weekly os package updates. During the last update, we noticed that the yarn build no longer produces a successful build. It completes successfully, and seems to start up fine, but trying to access via browser results in a 404. This is with 4.0.2. Interestingly, previous week's build with 4.0.2 had no issues. The only changes between the two builds are a few os packages updated (don't seem to be that relevant) and some npm packages used in the build. That is the main reason I started looking into utilizing the standalone packages you guys publish rather than building it with yarn. And that's when I noticed these issues I mentioned above.

jsjoeio commented 2 years ago

Right, qemu has some bugs with runtime, but builds are much more reliable.

TIL!

I'm thinking this part could be using qemu instead of cross-compile:

Ah, I think I understand. cc @code-asher @jawnsy since you two have more experience in this area. What do you think about this idea?

and that would also solve the argon2 issue.

That would be amazing!

During the last update, we noticed that the yarn build no longer produces a successful build. It completes successfully, and seems to start up fine, but trying to access via browser results in a 404. This is with 4.0.2. Interestingly, previous week's build with 4.0.2 had no issues. The only changes between the two builds are a few os packages updated (don't seem to be that relevant) and some npm packages used in the build.

That is odd indeed...Might have to investigate with @code-asher to see if we can figure out what the issue is. We have received an usually high number of 404 reports with 4.0.2. I've seen it a couple times but usually can't reproduce.

Thanks for bringing that up! We'll have to do some thorough testing before the next release is out. We have been publishing the latest main changes under the beta flag on npm. You can test this out with yarn global add code-server@beta. We're hoping to publish beta to Docker in the future too so folks like yourself can test releases earlier.

code-asher commented 2 years ago

qemu seems like it would be stabler, definitely down to switch

zarekvane commented 2 years ago

I successfully installed code-server in debian that is installed in termux proot environment with the following command, but after I entered the login page and input my password, it showed me the 404 page.

yarn global add code-server

jsjoeio commented 2 years ago

I'm going to start working on this this week. I'll update this comment with my notes as I go.

TODOs

[ ] See if I can reproduce this with 4.1.0

aptalca commented 2 years ago

@jsjoeio Thanks for the reminder. I tested 4.1.0 tarballs on both arm32v7 and arm64v8, and no more errors with @parcel/watcher when accessing the gui.

This issue can be closed (unless you wish to continue the qemu build discussion here).

jsjoeio commented 2 years ago

@aptalca you rock! Thanks for testing it and reporting back.

This issue can be closed (unless you wish to continue the qemu build discussion here).

I'll open a separate issue to track that and link your comments. Thanks so much for your help!

coder / code-server