Formal Proving Language grammar not supported

bookofproofs commented 3 years ago

I'm not sure if this is really an issue. Using a self-published Atom package language-fpl (currently version 0.3.2), I'm getting the above error message from tree-sitter-syntax-visualizer when I open text files with the extension *.fpl. The tree-sitter-fpl grammar package is published at npm here.

Is there a convention I'm not aware of that I haven't followed yet to visualize the syntax tree using your visualizer?

aminya commented 3 years ago

That's because your language package isn't installed. You should run apm link . to add it to the packages directory and reload Atom

bookofproofs commented 3 years ago

I'm an Atom newbie, but Atom shows in its Settings that it is installed. Also, there is a folder .atom\packages\language-fpl in my home directory. Restarting Atom doesn't help either.

aminya commented 3 years ago

Does your language package follow the template (e.g. adding the file extensions)? Have you built it?

Here is the full build command:

node-gyp configure && node-gyp build && apm rebuild

bookofproofs commented 3 years ago

Thank you so much for your help! I feel like a beginner (who with respect to Atom, npm, and tree-sitter I definitely am.)

My template is called fpl.cson and it contains currently and initially only

'name': 'Formal Proving Language'

'scopeName': 'source.fpl'

'type': 'tree-sitter'

'parser': 'tree-sitter-fpl'

'fileTypes' : 'fpl'

'firstLineRegex': [ '.*' ]

'folds': [
    {
        type: ['TheoryBlock', 'NamespaceBlock']
    }
    {
        start: {type: 'LeftBrace', index: 0},
        end:  {type: 'RightBrace', index:-1}
    }
]

Before publishing language.fpl, I followed step-by-step the description at creating-a-grammar/

Coming back to your last suggestion, I had to install node-gyp:

D:\Atom>npm install -g node-gyp
C:\Users\eite\AppData\Roaming\npm\node-gyp -> C:\Users\eite\AppData\Roaming\npm\node_modules\node-gyp\bin\node-gyp.js
+ node-gyp@8.1.0
added 91 packages from 30 contributors in 8.556s

Unfortunately, your first command runs in my environment into an error:

D:\Atom>cd language-fpl
D:\Atom\language-fpl>node-gyp configure
gyp info it worked if it ends with ok
gyp info using node-gyp@8.1.0
gyp info using node@14.17.1 | win32 | x64
gyp info find Python using Python version 3.8.0 found at "C:\Python\python.exe"
gyp info find VS using VS2019 (16.9.31112.23) found at:
gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community"
gyp info find VS run with --verbose for detailed information
gyp info spawn C:\Python\python.exe
gyp info spawn args [
gyp info spawn args   'C:\\Users\\eite\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'D:\\Atom\\language-fpl\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\eite\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\eite\\AppData\\Local\\node-gyp\\Cache\\14.17.1\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\eite\\AppData\\Local\\node-gyp\\Cache\\14.17.1',
gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\eite\\AppData\\Roaming\\npm\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\eite\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\14.17.1\\\\<(target_arch)\\\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=D:\\Atom\\language-fpl',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'D:\\Atom\\language-fpl\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp: binding.gyp not found (cwd: D:\Atom\language-fpl) while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\eite\AppData\Roaming\npm\node_modules\node-gyp\lib\configure.js:353:16)
gyp ERR! stack     at ChildProcess.emit (events.js:375:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12)
gyp ERR! System Windows_NT 10.0.19041
gyp ERR! command "D:\\Apps\\nodejs\\node.exe" "C:\\Users\\eite\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure"
gyp ERR! cwd D:\Atom\language-fpl
gyp ERR! node -v v14.17.1
gyp ERR! node-gyp -v v8.1.0
gyp ERR! not ok

D:\Atom\language-fpl>

bookofproofs commented 3 years ago

By the way, the code folding does not work either.

aminya commented 3 years ago

You should first generate the tree-sitter grammar, then run the build command I gave inside your tree-sitter-fpl package. Then once .node file is generated, you should link that package to Atom's language-fpl.

See these two packages to understand the workflow:

npm install && node-gyp configure && node-gyp build && apm rebuild

https://github.com/tree-sitter/tree-sitter-python


npm install path_to_tree-sitter-python
apm install
apm link .

https://github.com/atom/language-python

bookofproofs commented 3 years ago

Your script worked great to the point apm install where I got the message

Installing modules failed
npm WARN tarball tarball data for tree-sitter-fpl@1.0.1 (sha512-WKLKQSV/neCmBr7OMegSEK/mHZ/NiUT0Fhr19s32me4r4zJruJy8HCUcXTeTCjDx4TjTwNokvbXDpMin2JVHlg==) seems to be corrupted. Trying one more time.
npm WARN tarball tarball data for tree-sitter-fpl@1.0.1 (sha512-WKLKQSV/neCmBr7OMegSEK/mHZ/NiUT0Fhr19s32me4r4zJruJy8HCUcXTeTCjDx4TjTwNokvbXDpMin2JVHlg==) seems to be corrupted. Trying one more time.
npm ERR! code EINTEGRITY
npm ERR! Verification failed while extracting tree-sitter-fpl@1.0.1:
npm ERR! Verification failed while extracting tree-sitter-fpl@1.0.1:
npm ERR! sha512-WKLKQSV/neCmBr7OMegSEK/mHZ/NiUT0Fhr19s32me4r4zJruJy8HCUcXTeTCjDx4TjTwNokvbXDpMin2JVHlg== integrity checksum failed when using sha512: wanted sha512-WKLKQSV/neCmBr7OMegSEK/mHZ/NiUT0Fhr19s32me4r4zJruJy8HCUcXTeTCjDx4TjTwNokvbXDpMin2JVHlg== but got sha512-qDgee2G8PJ900z6lafYlMKNOz//MFs+9lht38gi9TV6K385mk1UMtUV7aZVETBq63ebPauBw+9xGve0ANnXv+Q==. (2721736 bytes)

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\eite\.atom\.apm\_logs\2021-06-30T12_24_42_363Z-debug.log

So I first thought I have to republish tree-sitter-fpl under a new version (1.0.2) which I did. But the step apm publish runs into the same "checksum" integrity error.

Then I thought I have to republish language-fpl (which I now republished to 0.3.3). Now, the step apm publish runs into the same "checksum" integrity error.

It's frustrating.

aminya commented 3 years ago

Are you sure you published it correctly? You seem to be including some files in the package that are not required, as the size is 13 MB! For testing this locally, you don't need to publish it on npm. Just use npm install path_to_package.

bookofproofs commented 3 years ago

How do I know I didn't? I do not understand why npm confirms publishing something without the required sanity checks. Do you know a good step-by-step intro on the Internet? I know only http://tree-sitter.github.io/tree-sitter/creating-parsers, https://flight-manual.atom.io/hacking-atom/sections/creating-a-grammar/ and https://flight-manual.atom.io/hacking-atom/sections/publishing/, but none of them describe the steps as you did above. The configure and build steps are not included in these descriptions. Also, they only mention npm, not apm, and node-gyp.

I'm afraid, I have to give up the combination Atom/tree-sitter/ npm and try something else, maybe a language server with another IDE. I heard about Atom and tree-sitter and wanted to give the tools a chance but they are too complicated for me or I'm too foolish to achieve "simple" results like syntax highlighting or code folding for the new language I created. Anyway, thank you very much for your kind assistance!

aminya commented 3 years ago

Creating tree sitter packages is complex, but once you make it working is not that hard. Did you try npm install path_to_package --save at the root of the atom package?

bookofproofs commented 3 years ago

Just to make sure, I'm not on the right path:

path_to_package - Which package do you mean? tree-sitter-fpl package at npm or the language-fpl Atom package?

aminya commented 3 years ago

When you want to test a package locally, you can use npm install path_to_package --save to do so.

atom-language-fpl requires tree-sitter-fpl. So you need to run npm install --save path_to_tree_sitter_fpl at the root of atom-language-fpl.

bookofproofs commented 3 years ago

Thank you. Which do you mean by

atom-language-fpl?

Here is my local file structure:

1) D:\Atom\language-fpl (local repo of the atom package) 2) D:\Atom\tree-sitter-fpl (local repo of the tree-sitter package) 3) C:\~\.atom\packages\language-fpl (local atom package installed with Atom)

npm install --save D:\Atom\tree-sitter-fpl worked in the 1st. directory but not in the 3rd.

What I still do not understand, if you mean 1st (what I expect), how can I

test a package locally

without having the package installed in Atom? I suppose I have now to republish the package so I can install it in Atom?! By "testing" I mean I could change for instance the theme or tweak the C:\Users\~\.atom\packages\language-fpl\grammars\fpl.cson file in my home directory during a running Atom session. When the tests are successful, I would then correct my 1st repo, commit, push and republish the Atom package so other users could use it.

My tree-sitter package works locally and so I published it already. I've got a functioning parser. What doesn't work, ist the Atom package, respectively tree-sitter-syntax-visualizer says it does not support its grammar.

bookofproofs commented 3 years ago

I just discovered this question in a tree-sitter-discussion. Maybe the simple reason for all my problems is that Atom does not yet support the newest tree-sitter I used to create my tree-sitter package.

tmatijevich commented 3 years ago

Atom language package:

Run apm links
- This will show if you have a link (Symlink/Junction) in %userprofile%\.atom\packages to D:\Atom\language-fpl
- If nothing is listed, %userprofile%\.atom\packages\language-fpl was installed from apm (where you published it)
If it's not linked, run apm link -d in D:\Atom\language-fpl to link your current repository to .atom\dev\packages
- You should probably disabled your original installed package in settings
You can (and should) run Atom with your packages before publishing, running in developer mode atom -d allows you to test new packages while keeping atom clean

Tree-sitter parser:

Your Atom package depends on this
Test your developing tree-sitter package in your developing atom package
- Run npm install --save D:\Atom\tree-sitter-fpl in D:\Atom\language-fpl
- Run atom -d

bookofproofs commented 3 years ago

Thank you for your more comprehensive instructions. I'll try.

Xapphire13 / tree-sitter-syntax-visualizer

Formal Proving Language grammar not supported #11