ts-safeql / safeql

Validate and auto-generate TypeScript types from raw SQL queries in PostgreSQL.
https://safeql.dev
MIT License
1.35k stars 22 forks source link

SafeQL installation fails on Windows #80

Closed Eprince-hub closed 7 months ago

Eprince-hub commented 2 years ago

Describe the bug SafeQL can not be installed on Windows 10 / 11. This is due to one of the dependencies, libpg-query that doesn't currently support installation on windows machines according to the issue here https://github.com/pyramation/libpg-query-node/issues/22

Running yarn add --dev @ts-safeql/eslint-plugin libpg-query throws the following errors

$ yarn add --dev @ts-safeql/eslint-plugin libpg-query
yarn add v1.22.19
[1/4] �  Resolving packages...
warning Resolution field "eslint-plugin-jest@27.0.4" is incompatible with requested version "eslint-plugin-jest@^25.3.0"
[2/4] �  Fetching packages...
[3/4] �  Linking dependencies...
warning "@emotion/react > @emotion/babel-plugin@11.10.2" has unmet peer dependency "@babel/core@^7.0.0".
warning "@emotion/react > @emotion/babel-plugin > @babel/plugin-syntax-jsx@7.18.6" has unmet peer dependency "@babel/core@^7.0.0-0".
warning " > @babel/eslint-parser@7.19.1" has unmet peer dependency "@babel/core@>=7.11.0".
warning " > eslint-plugin-flowtype@8.0.3" has unmet peer dependency "@babel/plugin-syntax-flow@^7.14.5".
warning " > eslint-plugin-flowtype@8.0.3" has unmet peer dependency "@babel/plugin-transform-react-jsx@^7.14.9".
[4/4] �  Building fresh packages...
[-/4] ⠁ waiting...
[-/4] ⠁ waiting...
[-/4] ⠂ waiting...
error C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build
Arguments:
Directory: C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.10
node-pre-gyp info using node@18.5.0 | win32 | x64
node-pre-gyp info check checked for "C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\Release\queryparser.node" (not found)
node-pre-gyp http GET https://supabase-public-artifacts-bucket.s3.amazonaws.com/libpg-query-node/queryparser-v13.2.5-node-v108-win32-x64.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://supabase-public-artifacts-bucket.s3.amazonaws.com/libpg-query-node/queryparser-v13.2.5-node-v108-win32-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not installable for libpg-query@13.2.5 and node@18.5.0 (node-v108 ABI, unknown) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error response status 404 Not Found on https://supabase-public-artifacts-bucket.s3.amazonaws.com/libpg-query-node/queryparser-v13.2.5-node-v108-win32-x64.tar.gz
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@18.5.0 | win32 | x64
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@18.5.0 | win32 | x64
gyp info find Python using Python version 3.9.6 found at "C:\Users\Victor\AppData\Local\Programs\Python\Python39\python.exe"
gyp info find VS using VS2017 (15.9.28307.1585) found at:
gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools"
gyp info find VS run with --verbose for detailed information
gyp info spawn C:\Users\Victor\AppData\Local\Programs\Python\Python39\python.exe
gyp info spawn args [
gyp info spawn args   'C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\libpg-query\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\Victor\\AppData\\Local\\node-gyp\\Cache\\18.5.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\Victor\\AppData\\Local\\node-gyp\\Cache\\18.5.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\Victor\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\18.5.0\\\\<(target_arch)\\\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\libpg-query',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\libpg-query\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@18.5.0 | win32 | x64
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe
gyp info spawn args [
gyp info spawn args   'build/binding.sln',
gyp info spawn args   '/clp:Verbosity=minimal',
gyp info spawn args   '/nologo',
gyp info spawn args   '/p:Configuration=Release;Platform=x64'
gyp info spawn args ]
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  nothing.c
  win_delay_load_hook.cc
  nothing.vcxproj -> C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\Release\\nothing.lib
  prebuild_dependencies
  The system cannot find the path specified.
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(209,5): error MSB6006: "cmd.exe" exited with code 1. [C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\queryparser.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\node-gyp\lib\build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:537:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Windows_NT 10.0.22000
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\node-gyp\\bin\\node-gyp.js" "build" "--fallback-to-build"
"--module=C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\libpg-query\\build\\Release\\queryparser.node" "--module_name=queryparser" "--module_path=C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\libpg-query\\build\\Release" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v108"
gyp ERR! cwd C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query
gyp ERR! node -v v18.5.0
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\Release\queryparser.node --module_name=queryparser --module_path=C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\@mapbox\node-pre-gyp\lib\util\compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:537:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1091:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:302:5)
node-pre-gyp ERR! System Windows_NT 10.0.22000
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Victor\\projects\\debugging-programs\\next-js-ecommerce-store\\node_modules\\@mapbox\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query
node-pre-gyp ERR! node -v v18.5.0
node-pre-gyp ERR! node-pre-gyp -v v1.0.10
node-pre-gyp ERR! not ok
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=C:\Users\Victor\projects\debugging-programs\next-js-ecommerce-store\node_modules\libpg-query\build\Release\queryparser.node --module_name=queryparser --module_path=C:\Users\Victor\projects\debugging-programs\nex

To Reproduce You can try this installation on this repo A computer running windows OS is needed to reproduce this error

clone project

git clone https://github.com/Eprince-hub/libpg-query-installation-error.git

cd into project

cd libpg-query-installation-error

install SafeQL

yarn add --dev @ts-safeql/eslint-plugin libpg-query
or
npm install --save-dev @ts-safeql/eslint-plugin libpg-query

Expected behaviour Install SafeQL on a windows computer without an issue

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

Newbie012 commented 2 years ago

That's not exactly an issue of SafeQL, but libpg-query.

One of my machines is using Windows, but all of the development occurs in WSL (Windows Subsystem for Linux) and it seems to work fine.

Another workaround that I can think of is using Docker.

Newbie012 commented 2 years ago

I'll take a look at https://github.com/taozhi8833998/node-sql-parser as well

Newbie012 commented 2 years ago

I ran some tests, and it looks like node-sql-parser isn't working as expected. It fails to parse many syntaxes such as:

SELECT ((colname + INTERVAL '1 month')::DATE - 1) as x

If I change it to this (which I believe will result in a runtime error; haven't checked):

SELECT (colname + INTERVAL '1 month')::DATE - 1 as x

it will throw an error.

I believe we should stick with libpg-query since it uses the real PostgreSQL query parser. Let's see how the conversation in https://github.com/pyramation/libpg-query-node/issues/22 evolves.

swax commented 1 year ago

Any chance you could mention that this doesn't work on Windows on your website and/or the readme file? Just spent a good amount of time trying to figure out why this wasn't installing.

I am looking forward to using it someday!

Newbie012 commented 1 year ago

@swax done - https://github.com/ts-safeql/safeql#limitations

karlhorky commented 1 year ago

@gregnr opened a PR using emnapi + WebAssembly to get libpg_query support on Windows 👀 🔥 (in the libpg-query npm package)

Newbie012 commented 10 months ago

Hopefully should be resolved if https://github.com/Hilzu/libpg-query-node/issues/11 will be resolved as well.

karlhorky commented 10 months ago

Thanks to @lfittl libpg_query@16-5.1.0 now has support for compiling on Windows too 🎉 🚀

karlhorky commented 8 months ago

There's a PR by @aquariuslt open in libpg-query now 👀 🙌

karlhorky commented 7 months ago

The PR has been merged and published as libpg-query@16.1.0 - installable on Windows, finally!

https://github.com/launchql/libpg-query-node/pull/45#issuecomment-2026913675

karlhorky commented 7 months ago

@Eprince-hub can you confirm that SafeQL performs checking and shows errors and warnings on Windows now?

karlhorky commented 7 months ago

Just trying out the new libpg-query@16.1.0 Windows version on a GitHub Actions windows-latest runner, and it looks like there's a new ERR_UNSUPPORTED_ESM_URL_SCHEME error with SafeQL regardless of whether a linting problem is encountered or not:

pnpm eslint . --max-warnings 0

Oops! Something went wrong! :(
ESLint: 8.57.0
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:239:11)
    at defaultLoad (node:internal/modules/esm/load:130:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:409:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:291:56)
    at new ModuleJob (node:internal/modules/esm/module_job:65:26)
    at #createModuleJob (node:internal/modules/esm/loader:303:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:260:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:241:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:328:23)
Error: Process completed with exit code 1.

This appears to be related to the Node.js requirement on Windows to "need to write file:///c:/x/y/z instead of c:/x/y/z"

There are no ESM errors or warnings from the ESLint run on Windows when SafeQL is disabled:

Screenshot 2024-04-03 at 10 12 16

So this appears to be a SafeQL-specific bug with Windows support.

cc @Newbie012

karlhorky commented 7 months ago

Actually, looking at this closer and trying to create a reproduction, it looks like it's something else in eslint-config-upleveled that's causing it, since a simple SafeQL config seems to work:

https://github.com/upleveled/preflight-test-project-next-js-passing/pull/156#issuecomment-2038342377

Edit: Yes confirmed, the problem with ERR_UNSUPPORTED_ESM_URL_SCHEME was an error in our own eslint-config-upleveled package:

It seems like using process.cwd() in a dynamic import() doesn't work on Windows, have to use pathToFileURL:

+import { pathToFileURL } from 'node:url';

-await import(`${process.cwd()}/package.json`, {
+await import(pathToFileURL(`${process.cwd()}/package.json`).href, {
  assert: { type: 'json' },
});
karlhorky commented 7 months ago

I have now successfully been able to receive a SafeQL error on GitHub Actions, on a Windows runner 🎉

  await sql`
    INSERT INTO
      loogos (name)
    VALUES
      ('Next.js'),
      ('Next.js'),
      ('Next.js'),
      ('Next.js')
  `;

SafeQL error:

$ pnpm eslint . --max-warnings 0
(node:2004) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

D:\a\preflight-test-project-next-js-passing\preflight-test-project-next-js-passing\migrations\00000-create-table-logos.ts
Error:   25:7  error  Invalid Query: relation "loogos" does not exist  @ts-safeql/check-sql

✖ 1 problem (1 error, 0 warnings)

Error: Process completed with exit code 1.

While testing the Windows SafeQL issue, I created a cross-platform PostgreSQL + SafeQL reproduction repo (Windows, macOS and Linux runners).

Now that I solved it, I thought this may also be a useful thing for users of the SafeQL project longer term (eg. for users demonstrating their bugs):

cc @Newbie012 , in case we should integrate this into the issue workflow on GitHub.

karlhorky commented 7 months ago

@Eprince-hub if you could do one more check from your side on your Windows system, then I think this issue can be closed.

Eprince-hub commented 7 months ago

@karlhorky SafeQL linting error message on Windows Computer

Screenshot 2024-04-05 152511

Because of the issue with a long path not being found by node on Windows: https://github.com/nodejs/node/issues/50753

I needed to run the two scripts below to get it working

echo 'node-linker=hoisted' > ./.npmrc
pnpm install --force

I will be closing this issue as it is now confirmed that SafeQL can be installed and used on Windows Machines