Use Syntax instead of ctags?

[mkleehammer]

I believe it would be not be too difficult to generate the tags from the atom/textmate syntax files themselves which has a lot of benefits:

1. Every file type that has a syntax could now have symbols generated automatically. The ctags project is never going to be able to keep up with all of the great new syntaxes as Atom grows.
2. Hacking ctags is much more difficult than atom packages, further impeding improvements.
3. Performance for Toggle File Symbols should be better since open files are already being tokenized.
4. This would eliminate an entire dependency that requires special handling on Windows, etc. Grammars, first-mate, etc. are already part of the project.
5. Less fragile since it would be using other Atom syntax settings and code. (For example, the mapping from syntax to ctags file types bug would not have happened if the actual syntax was being used.)

I performed some initial testing using grammar.tokenizeLines and scanning the results for items like meta.function and it seems to work well. (Well, coffee script cuts off the last letter of a function name and makes a separate function from it, but that is easy to work around until it is fixed in first-mate.)

[dyross]

:+1: ctags won't get as good performance as ST3.

[nathansobo]

I'd definitely be open to something like this. Ctags was expedient early on when we needed to get something working quickly. Presumably this is something that could first be built as an external library that depends on [first-mate][first-mate] for parsing, then we could populate an index from a background task. There's a lot on our list ahead of this, but if someone wants to start on a library / command line utility I'm all for it.

[dyross]

It looks like @mkleehammer started something like this, but hasn't worked on it for a bit: https://github.com/mkleehammer/goto

[mkleehammer]

Actually I use it every day I just haven't had any need to change anything lately. I like it but a lot of javascript code uses var NewCls = Backbone.Model.extend() type of code to create classes and those are marked as simple variables by the syntax. I might have to add some brute force to it for certain syntaxes. Same with languages that support overloaded functions so we can see the parameters.

[nathansobo]

Is it indexing in process or out of process?

[kemayo]

ST3 does this, as I understand it, and it's wonderfully hands-off from the end-user perspective.

[dyross]

To me, this is the only major feature that Sublime has but Atom doesn't. But I need it to use Atom regularly.

[yongkangchen]

I try goto package, and it indexing in process (@nathansobo ), so project indexing will block atom.

For project tags, I prefer ctags, and I suggest you guys try my atom-ctags package.

As @mkleehammer said: Ctag never going to be able to keep up is wrong, I implement auto update file's tag when save.

For file tags, I agree that Performance for Toggle File Symbols should be better since open files are already being tokenized

[mkleehammer]

mkleehammer (me) did not say that- if you re-read the complete sentence you will see I suggested that ctags is not going to be able to support new Atom syntaxes (languages) as quickly as they are written, if at all. As an example, Atom already supports Apple's Swift language and exuberant ctags does not. I'm sure it will because Apple is huge, but what if other languages like Rust, closure, etc.?

You've written a hook to make incremental updates of a ctags database. Well done. However it is not related to any of my points which are about languages supported by ctags.

[yongkangchen]

As an example, Atom already supports Apple's Swift language and exuberant ctags does not. Can quick support, just if anyone who like to do, and know how to do. And this is a example how I enhance ctags support Lua:https://gist.github.com/yongkangchen/10120546

Write a ctags regex langdef is not much harder than you think.

[yongkangchen]

I search for Rust langdef for ctags: https://github.com/rust-lang/rust/blob/master/src/etc/ctags.rust

And also to be specific, that ctags is a command tool which generate tag file. Any tool can generate tag with the same format. For closure, https://github.com/AnyChart/gjstags, this tools generate the ctag's tag file.

And this https://github.com/atom/symbols-view/blob/master/lib/.ctags is how atom support CoffeeScript and so on

[yongkangchen]

I would like your goto package more, if you resolved project symbol generate too slow to block atom. And can support symbol Persistence to disk file. After you do these, Why just use ctags directly? Of course, I do specific my point again: 1、For project tags, I prefer ctags. 2、For file tags, I prefer Syntax.

[mkleehammer]

I am apparently not making my point, yet you are almost making it for me. Swift is not supported. Rusr isn't in my copy on OS X, Actually compare the list of syntaxes to the supported ctags languages.

"and know how to d" (sic) is precisely my point. Yes, I could add language support to ctags and you can too, but very few people do. I could edit in nano, but there are reasons I use Atom and emacs. Just because something can do a job doesn't mean there aren't other ways or tools with additional benefits.

A big draw of Atom is that almost all of the stack is in JavaScript and packages can be downloaded. Using ctags adds a significant impedance mismatch. Similarly, one already has to learn the TextMate syntax mechanism to add language support to Arom. Using ctags means there are now two syntaxes.

It seems obvious that using a single source for a language definition is better than using two. It seems to me that having all aspects of language support be in a single place is also better. More importantly, it would be complete automatic.

I have little doubt that in the long term there will be more Eclipse like scaffolding to support full project indexing. I am suggesting we start now

I ne could suggest an automatic mapping from Atom/TextMate syntax to ctags regexps. I would still think it falls far short of a JS only solution, but it would be a major step forward.

I've used emacs and ctags for many years, but Atom has a different style and ctags seems like it doesn't fit well.

[dyross]

I wonder why we aren't seeing atom/core people involved in these features?

[yongkangchen]

@mkleehammer, I totally got it. @dyross, first today is not a working day. Also, their focus is not on this. but the https://github.com/atom/atom/issues/2448

We currently have three main initiatives:
* Make the react editor the default editor #2423
* Basic, better windows support for developers #2447
* Issues and PRs

[Trudko]

any update on this?