search

orzechowskid / tree-sitter-css-in-js

tree-sitter grammar for CSS-in-JS
Other
10 stars 5 forks source link

Build script cannot find grammar module #19

Open stevemolitor opened 2 years ago

stevemolitor commented 2 years ago

Thanks for working on this. This seems a like a great solution to the CSS in JS problem.

A few issues I encountered, the last being a blocker for me:

No M1 silicon binary

I'm using a native silicon build of Emacs for my M1 mac, so the css_in_js.so binary file I extracted from the macos.tar.gz file in the release failed to load:

File mode specification error: (tsc-lang-load-failed dlopen(/Users/stephenmolitor/.emacs.d/straight/build/tree-sitter-css-in-js/css_in_js.so, 0x0005): tried: '/Users/stephenmolitor/.emacs.d/straight/build/tree-sitter-css-in-js/css_in_js.so' (not a mach-o file))

git clone fails if you have github SAML auth setup

I tried building the binary myself but the git clone command run in postinstall doesn't work for me because my organization requires SAML auth. I worked around that my changing the clone URL to git@github.com:orzechowskid/tree-sitter-css-in-js.git, to use the SSH transfer protocol.

However I might be missing the big picture here - I don't know why it has to clone a copy of itself. Maybe some build instructions might be nice, for those whose architecture is not pre-built.

Cannot run npm install twice

This is a nit, but I also noticed that if I re-run npm install it gives an error complaining that the the clone directory already exists:

> tree-sitter-css-in-js@0.0.1 postinstall
> npm run clone

> tree-sitter-css-in-js@0.0.1 clone
> git clone git@github.com:orzechowskid/tree-sitter-css-in-js.git

fatal: destination path 'tree-sitter-css-in-js' already exists and is not an empty directory.
npm ERR! code 128
npm ERR! path /Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js
npm ERR! command failed
npm ERR! command sh -c npm run clone

It'd be a bit nicer if it checked for the existence of the directory first before trying to clone again. But this is not important.

Build script cannot find grammar module

This was the blocker for me. Running npm run build from the root directory produces this error for me:

node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module './tree-sitter-css/grammar'
Require stack:
- /Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js/grammar.js
- /Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js/[stdin]
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js/grammar.js:9:26)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js/grammar.js',
    '/Users/stephenmolitor/.emacs.d/straight/repos/tree-sitter-css-in-js/[stdin]'
  ]
}
Node process exited with status 1
orzechowskid commented 2 years ago

Build instructions are a great idea. I was hoping that nobody would have to build anything in the first place, but I don't think that's going to be the case :)

stevemolitor commented 2 years ago

@orzechowskid Derp yeah I was cloning the wrong repo my bad!

Changing this line in package.json to use the git@ URL fixed the clone problem for me: git@github.com:tree-sitter/tree-sitter-css.git. It prompted me to enter my passphrase, and cloned tree-sitter-css successfully.

~I'm still not sure how to build. I tried npm run build and cargo build --release (I have rust and npm installed), and it did stuff apparently successfully, but I couldn't find any .so (nor .dylib) files.~

EDIT: I found the .so file in ~/Library/Caches/tree-sitter/lib/css_in_js.so, and copied it over to ~/.emacs.d/straight/build/tree-sitter-css-in-js. However I still get this error from emacs:

File mode specification error: (tsc-lang-load-failed dlopen(/Users/stephenmolitor/.emacs.d/straight/build/tree-sitter-css-in-js/css_in_js.so, 0x0005): tried: '/Users/stephenmolitor/.emacs.d/straight/build/tree-sitter-css-in-js/css_in_js.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))))

I wonder if it build a Rosetta lib or something? Not really sure.

orzechowskid commented 2 years ago

oh, thank you for that git url! I'll make that change when I get a minute.

it's my understanding that elisp-tree-sitter should already be scanning ~/Library/Caches/tree-sitter/lib for shared-library objects to use as language parsers, and as you've discovered that is the default output directory for the tree-sitter-cli tool, so you shouldn't need to manually copy anything. C-h v tree-sitter-load-path should show you exactly where elisp-tree-sitter is looking, if that's helpful too.

re: architecture mismatch, I'm afraid I'm not sure what's going on. I don't have an M1 machine to play with. I did notice though that node.js offers two different download options for Mac machines: x86_64 and arm64. is it possible that you're using x86_64 node.js via Rosetta? I'm a little out of my depth here but I wonder if having the "wrong" version of node and its header files would result in shared libraries compiled for an architecture other than the one you want. sorry I can't be of more help here!

stevemolitor commented 2 years ago

Actually ~/Library/Caches/tree-sitter/lib was not in my tree-sitter-load-path. But I added it, same result. 😞

I did notice though that node.js offers two different download options for Mac machines: x86_64 and arm64. is it possible that you're using x86_64 node.js via Rosetta? I'm a little out of my depth here but I wonder if having the "wrong" version of node and its header files would result in shared libraries compiled for an architecture other than the one you want.

That's very possible! I will check into that later.

The error message says ... have (x86_64), need (arm64e). So yeah somehow it's building for the wrong architecture. I'll play around with it more in a day or two.

stevemolitor commented 2 years ago

Update:

I downloaded and installed the arm64 version of node, but no luck.

Running file confirms I have the wrong architecture:

file css_in_js.so
css_in_js.so: Mach-O 64-bit dynamically linked shared library x86_64

Running file on vterm's so file shows it is an arm64 file:

file vterm-module.so
vterm-module.so: Mach-O 64-bit bundle arm64

I played around with various --arch options to node-gyp, to no avail. But I too am a little out of my depth here. I'm not really sure how to tell node-gyp to build an arm64 shared lib. I may take a look at vterm and see what it is doing.

I'm not sure where my other tree-sitter parser libs are located - there were no other .so files under my ~/.emacs.d or ~/Library/Cache/tree-sitter. I wonder how and where they are building.

stevemolitor commented 2 years ago

OK I got it working by following the instructions in this comment:

tree-sitter generate
gcc -o parser.so --shared src/parser.c src/scanner.c -I./src
mv parser.so ~/Library/Caches/tree-sitter/lib/css_in_js.dylib

(I had added ~/Library/Caches/tree-sitter/lib to my tree-sitter-load-path earlier.)

I can confirm that tsx-mode issues 11 and 15 are now fixed! 🥳

stevemolitor commented 2 years ago

I am running into a weird issue though. Frequently emacs will lock up upon loading a .tsx file now. If I revert back to tsx-mode master branch it does not lock up.

If I open a tsx file straight away upon emacs, without opening a file in any other mode (except *scratch* which I have configured to use fundamental mode), everything works fine, and stays fine as I open and close files of different types.

However, if I open say a .el file or a markdown file first, and then open a file that starts tsx-mode, it locks up. I can't enter any text. If I press C-g everything is fine again, and I see this in the *Messages* buffer:

Error during redisplay: (jit-lock-function 1) signaled (quit)

I started a rosetta emacs I have installed, and rebuilt everything and used the original x86_64 css_in_js.so file, and had the same problem. Switching back to the master branch of tsx-mode and not using tree-sitter-css-in-js fixes the problem. So it doesn't seem like an architecture (arm64 vs x86_64) issue. I suspect that the new grammar is causing something to lock up, but don't know for certain.

EDIT: I am using Emacs 29 builds. Busy with work atm, but I will try with 28 and create a proper issue in a day or two.

orzechowskid commented 2 years ago

progress! that's great to hear!

last night I dug into the tree-sitter-langs metapackage a little and it seems like shared-library compilation is performed via some elisp which invokes a compiler via make-process. I also found this comment, which seemed interesting: https://github.com/emacs-tree-sitter/tree-sitter-langs/blob/5c6900a66a6b3a5d4ae6bde5199b5288c09af43b/tree-sitter-langs-build.el#L345 . I wonder if I could do something similar in a Github Action, or if producing an arm64 shared library really will require some steps to be taken manually by users with M1 machines.

that lockup issue is really strange though. what happens if you (setq debug-on-quit t) and then reproduce the problem? hopefully that will let you see where things are stuck.

stevemolitor commented 2 years ago

Yeah I saw that comment re the Silicon compile workaround in tree-sitter-langs too. That clued me in that maybe I had to compile things differently. I did try building via the Add a new grammar instructions in tree-sitter-langs. I ran into some problem like an eslip require problem when running the tree-sitter-langs scripts that use cask - don't recall exactly but maybe easy enough to solve. It'd be cool if you could hook into tree-sitter-langs build process, as they apparently handle the Silicon workaround for you.

I noticed that tree-sitter-langs builds all the shared libs for me under .emacs.d/straight/build/tree-sitter-langs/bin/*.dylib. (I missed that earlier because I was looking for .so files.) That's cool.

I'll try (setq debug-on-quit t).

stevemolitor commented 2 years ago

Doing (setq debug-on-quit t) didn't reveal anything. But using the built-in profiler did - flyspell-post-command-hook -> flyspell-word was taking up a big chunk of memory and a decent amount of CPU.

Instead of (add-hook 'prog-mode-hook 'flyspell-mode) I'm now adding a hook that starts flyspell-mode in a run-with-idle-timer callback after one second of idle time. That resolves the issue.

I'm still not quite sure why this was only a problem when using tree-sitter-css-in-js. However I have an acceptable (to me) work around, and I'm a happy camper now.

I did notice a minor indentation issue on blank lines in CSS in JS blocks, which I fixed in tsi.el (just a quick hack, no tests yet). However I think that might break things in the master branch of tsx-mode.el. 🤔