get-convex / convex-backend

The Convex open-source backend
https://docs.convex.dev
Other
2.08k stars 78 forks source link

Please consider build instructions for Windows #7

Open AddictArts opened 3 months ago

AddictArts commented 3 months ago

Hi, I am trying out convex; However, the justfile etc seem to heavily favor a unix environment. This project does provide Windows releases, so I was hoping to perform the same release build.

I tried cargo build -p local_backend --bin convex-local-backend

I get the following error;

error: failed to run custom build command for `isolate v0.1.0 (D:\box\rust\convex-backend\crates\isolate)`

Caused by:
  process didn't exit successfully: `D:\box\rust\convex-backend\target\debug\build\isolate-79d8b7682c466825\build-script-build` (exit code: 1)

...

--- stderr
  The system cannot find the path specified.
  Error: Failed to 'rush install'

Now that is debug, release same issue, Also the npm-packages is not installing on Windows either, performing rush install in npm-packages. Thanks for considering and any help.

nipunn1313 commented 3 months ago

Hi. Thanks for checking it out on windows!

You can see how the build on windows works here. It also uses Just, rush, all the same toolchain. We specify using the bash shell on windows (Git Bash).

https://github.com/get-convex/convex-backend/blob/main/.github/workflows/precompile.yml

I don't think it particularly "heavily favor a unix environment" given that those commands are directly used on windows in precompile.yml - but if you could clarify what you mean there, that could be helpful in debugging.

At least at a glance based on your error message, I would guess that you hadn't yet installed npm install --prefix scripts as that would install rush. Error message seems to indicate that rush isn't installed.

Would appreciate PRs with instructional improvements for Windows once you get it working if you are so inclined.

AddictArts commented 3 months ago

Thanks @nipunn1313 I will investigate more. npm install --prefix scripts does not work on Windows. Some sort of existing long standing issue with npm unfortunately.

From a CMD prompt:

D:\box\rust\convex-backend>npm install --prefix scripts
npm error code ENOENT
npm error syscall open
npm error path D:\box\rust\convex-backend\package.json
nipunn1313 commented 3 months ago

try from git bash

All of the scripts we run in our CI on windows are from git bash.

AddictArts commented 3 months ago

try from git bash

All of the scripts we run in our CI on windows are from git bash.

MINGW64 /d/box/rust/convex-backend (main)
$ npm install --prefix scripts
npm error code ENOENT
npm error syscall open
npm error path D:\box\rust\convex-backend\package.json

Wish it worked, but not from git bash for me either. Why the --prefix? can you just cd scripts && npm install?

nipunn1313 commented 3 months ago

can repro. Well that's bizarre

image

CI uses npm ci --prefix scripts which does appear to work

cd scripts ; npm install does work. No reason to prefer one vs the other.

I see that you've gone down the same research rabbithole as me and found https://github.com/npm/cli/issues/1290 https://github.com/npm/cli/issues/7722

Thanks for digging!

AddictArts commented 3 months ago

@nipunn1313 The github pipeline is using npm v8.x and npm is now v9.x. I have not tested with v8.x, but suspect the prefix may work in v8. It appears to me from review of the source and some build.rs the main point is to get node_modules in the scripts folder, so in effect --prefix is being used to eliminate having to change directories away from the root.

I am inclined to suggest a just recipe such as

pre-rush-install:
  cd scripts
  npm install

Or something like it and hopefully be able to use npm:latest and eliminate the somewhat obscure --prefix.

May also wish to check the npm version if v8 is really required, which I have not determined myself.

nipunn1313 commented 3 months ago

I don't think the npm version is relevant. --prefix seems to work with npm ci but not with npm i

We probably don't need to pin to v8 I'll take a look

AddictArts commented 3 months ago

@nipunn1313 thanks. In a git bash shell the next step of just rush install has the following error for me; However this is node v20 not v18 as git bash is not respecting the environment change fnm fast node manager nvm equiv on WIndows.

just rush install output in git-bash.

../../convex prepare: ...winpython\current\python-3.12.4.amd64\python.exe: can't open file 'D:\\box\\rust\\convex-backend\\npm-packages\\convex\\scripts\\scripts\\build.py': [Errno 2] No such file or directory
../../convex prepare: Failed
 ELIFECYCLE  Command failed with exit code 2.
../../id-encoding prepare: > id-encoding@0.1.2 build-esm
../../id-encoding prepare: > tsc

The command failed:
 D:\box\rust\convex-backend\npm-packages\common\temp\pnpm-local\node_modules\.bin\pnpm install --store D:\box\rust\convex-backend\npm-packages\common\temp/pnpm-store --config.cacheDir=D:\box\rust\convex-backend\npm-packages\common\temp/pnpm-store --config.stateDir=D:\box\rust\convex-backend\npm-packages\common\temp/pnpm-store --no-prefer-frozen-lockfile --no-strict-peer-dependencies --config.resolutionMode=highest --config.ignoreCompatibilityDb --recursive --link-workspace-packages false
ERROR: Error: Process exited with code 1
Giving up after 1 attempts

Was a little surprised to see pnpm in there, my node install has corepack enabled and pnpm.

nipunn1313 commented 3 months ago

It's gotta be node v18 (per the .nvmrc). I have it set up with nvm on windows and things work. You don't have to use nvm, but you do need to be on the version of node specified by .nvmrc

I was able to get just rush install working on git bash in my setup.

nipunn1313 commented 3 months ago

somehow there is this path D:\\box\\rust\\convex-backend\\npm-packages\\convex\\scripts\\scripts\\build.py which looks wrong - scripts is there twice in the path.

Not sure why it's happening, you may have to debug a bit. I do not face this issue on my setup.

nipunn1313 commented 3 months ago

https://github.com/get-convex/convex-backend/commit/82337bd96227bcd6256e90a37b69d84f8cbc1ba7 and https://github.com/get-convex/convex-backend/commit/5ccd4c9cbda4c4d3a75dbf5a0c770e3f13b3fb84

should work around the original two mentioned