Open liuchengxu opened 5 years ago
No stable way because the semantics of the scope field is not well defined. See https://github.com/universal-ctags/ctags/issues/2063
def __init__(self):
pass
class Baz(object):
def __init__(self):
def qux():
pass
What you get from ctags:
[yamato@slave]~/var/ctags-github% ./ctags --output-format=json --fields=s -o - /tmp/foo.py
{"_type": "tag", "name": "Baz", "path": "/tmp/foo.py", "scope": "Foo", "scopeKind": "class"}
{"_type": "tag", "name": "Foo", "path": "/tmp/foo.py"}
{"_type": "tag", "name": "__init__", "path": "/tmp/foo.py", "scope": "Foo.Baz", "scopeKind": "class"}
{"_type": "tag", "name": "__init__", "path": "/tmp/foo.py", "scope": "Foo", "scopeKind": "class"}
{"_type": "tag", "name": "qux", "path": "/tmp/foo.py", "scope": "Foo.Baz.__init__", "scopeKind": "member"}
[yamato@slave]~/var/ctags-github% cat foo.py
pattern fields are truncated to make reading easier. I should provide an option to hide pattern fields, BTW.
Step1. Pick an item which has no scope field from 5 items are in the list. Foo is the one. This one becomes the root of a tree. Step2. Pick items having the one found in the step1 in their scope fields. Foo is found in Baz and __init_. They are the second level branches of the root. Step3. Give a name to the branch. The upper-level branch name + "." + the current name is the branch name for the current branch. "Foo" + "." + "Baz" is for "Baz". "Foo" + "." + "init" is for "init". Step4. Pick items have the one found in the step2 in their scope fields. init__ is found in Foo.Baz. Is is the third level branches from the root. The parent is "Foo.Baz". ...
I wonder whether this algorithm is applicable to the other languages.
In some languages, scope fields are filled with full qualified names. In other languages, scope fields are filled with non-full qualified names. How do I say... "simple name".
Separators combine a parent name and the name of its child are not always '.'. e.g.: C++ uses '::' and '.'. pseudo tags are mechnism for notifying language specific separators from ctags to a client tool like vim. However, it is not well maintained.
Thanks for the response! I think the seperators used by various languages are not very big problem, Will try with your approach.
I should provide an option to hide pattern fields, BTW.
@masatake Let me know when you add this option, which will reduce some overhead in vista.vim as I currently have to keep the whole raw tag list in memory for vista.vim, although it's fine in most time.
It works.
$ git diff |cat
diff --git a/main/field.c b/main/field.c
index 1f9cbac6..f4905e23 100644
--- a/main/field.c
+++ b/main/field.c
@@ -28,9 +28,9 @@
#include "read.h"
#include "routines.h"
#include "trashbox.h"
+#include "writer_p.h"
#include "xtag_p.h"
-
typedef struct sFieldObject {
fieldDefinition *def;
unsigned int fixed: 1; /* fields which cannot be disabled. */
@@ -927,7 +927,7 @@ extern bool isFieldEnabled (fieldType type)
static bool isFieldFixed (fieldType type)
{
- return getFieldObject(type)->fixed? true: false;
+ return (writerHasFixedField() && getFieldObject(type)->fixed)? true: false;
}
extern bool enableField (fieldType type, bool state, bool warnIfFixedField)
@@ -1123,7 +1123,7 @@ static void fieldColprintAddLine (struct colprintTable *table, int i)
typefields[offset] = fieldDataTypeFalgs[offset];
}
colprintLineAppendColumnCString (line, typefields);
- colprintLineAppendColumnBool (line, fobj->fixed);
+ colprintLineAppendColumnBool (line, isFieldFixed (i));
colprintLineAppendColumnCString (line, fdef->description);
}
diff --git a/main/writer-json.c b/main/writer-json.c
index 68646f03..da579b0f 100644
--- a/main/writer-json.c
+++ b/main/writer-json.c
@@ -162,13 +162,17 @@ static void addExtensionFields (json_t *response, const tagEntryInfo *const tag)
static int writeJsonEntry (tagWriter *writer CTAGS_ATTR_UNUSED,
MIO * mio, const tagEntryInfo *const tag)
{
- json_t *pat = escapeFieldValue(tag, FIELD_PATTERN, true);
- json_t *response = json_pack ("{ss ss ss sO}",
- "_type", "tag",
- "name", tag->name,
- "path", tag->sourceFileName,
- "pattern", pat);
- json_decref (pat);
+ json_t *response = json_pack ("{ss}", "_type", "tag");
+
+ if (isFieldEnabled (FIELD_NAME))
+ json_object_set_new (response, "name", json_string (tag->name));
+ if (isFieldEnabled (FIELD_INPUT_FILE))
+ json_object_set_new (response, "path", json_string (tag->sourceFileName));
+ if (isFieldEnabled (FIELD_PATTERN))
+ {
+ json_t *pat = escapeFieldValue(tag, FIELD_PATTERN, true);
+ json_object_set_new (response, "pattern", pat);
+ }
if (includeExtensionFlags ())
{
diff --git a/main/writer.c b/main/writer.c
index a5bffb02..b5c027e5 100644
--- a/main/writer.c
+++ b/main/writer.c
@@ -96,3 +96,8 @@ extern bool writerCanPrintPtag (void)
{
return (writer->writePtagEntry)? true: false;
}
+
+extern bool writerHasFixedField (void)
+{
+ return (writer == &uCtagsWriter || writer == &eCtagsWriter)? true: false;
+}
diff --git a/main/writer_p.h b/main/writer_p.h
index 94c7ab36..d00fb7ad 100644
--- a/main/writer_p.h
+++ b/main/writer_p.h
@@ -71,5 +71,6 @@ extern bool ptagMakeJsonOutputVersion (ptagDesc *desc, void *data CTAGS_ATTR_UNU
extern bool ptagMakeCtagsOutputMode (ptagDesc *desc, void *data CTAGS_ATTR_UNUSED);
extern bool writerCanPrintPtag (void);
+extern bool writerHasFixedField (void);
#endif /* CTAGS_MAIN_WRITER_PRIVATE_H */
[jet@living]~/var/ctags% ./ctags -o - --output-format=json --fields=-PF main/main.c
./ctags -o - --output-format=json --fields=-PF main/main.c
{"_type": "tag", "name": "CLOCKS_PER_SEC", "file": true, "kind": "macro"}
{"_type": "tag", "name": "CLOCK_AVAILABLE", "file": true, "kind": "macro"}
{"_type": "tag", "name": "Totals", "typeref": "struct:__anonae81ef0f0108", "kind": "variable"}
{"_type": "tag", "name": "__anonae81ef0f0108", "file": true, "kind": "struct"}
{"_type": "tag", "name": "addTotals", "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "batchMakeTags", "file": true, "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "bytes", "file": true, "typeref": "typename:long", "kind": "member", "scope": "__anonae81ef0f0108", "scopeKind": "struct"}
{"_type": "tag", "name": "clock", "file": true, "kind": "macro"}
{"_type": "tag", "name": "clock", "file": true, "typeref": "typename:clock_t", "kind": "function"}
{"_type": "tag", "name": "createTagsForArgs", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsForEntry", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsForWildcardArg", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsForWildcardEntry", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsForWildcardUsingFindfirst", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsFromFileInput", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "createTagsFromListFile", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "etagsInclude", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "files", "file": true, "typeref": "typename:long", "kind": "member", "scope": "__anonae81ef0f0108", "scopeKind": "struct"}
{"_type": "tag", "name": "interactiveLoop", "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "isDestinationStdout", "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "isSafeVar", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "lines", "file": true, "typeref": "typename:long", "kind": "member", "scope": "__anonae81ef0f0108", "scopeKind": "struct"}
{"_type": "tag", "name": "main", "typeref": "typename:int", "kind": "function"}
{"_type": "tag", "name": "mainData", "file": true, "typeref": "typename:void *", "kind": "variable"}
{"_type": "tag", "name": "mainLoop", "file": true, "typeref": "typename:mainLoopFunc", "kind": "variable"}
{"_type": "tag", "name": "plural", "file": true, "kind": "macro"}
{"_type": "tag", "name": "printTotals", "file": true, "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "recurseIntoDirectory", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "recurseUsingOpendir", "file": true, "typeref": "typename:bool", "kind": "function"}
{"_type": "tag", "name": "runMainLoop", "file": true, "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "sanitizeEnviron", "file": true, "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "setMainLoop", "typeref": "typename:void", "kind": "function"}
{"_type": "tag", "name": "timeStamp", "file": true, "kind": "macro"}
[jet@living]~/var/ctags% ./ctags -o - --output-format=json --list-fields
./ctags -o - --output-format=json --list-fields
#LETTER NAME ENABLED LANGUAGE JSTYPE FIXED DESCRIPTION
C compact no NONE s-- no compact input line (used only in xref output)
E extras no NONE s-- no Extra tag type information
F input yes NONE s-- no input file
K NONE no NONE s-- no Kind of tag as full name
N name yes NONE s-- no tag name
P pattern yes NONE s-b no pattern
R NONE no NONE s-- no Marker (R or D) representing whether tag is definition or reference
S signature no NONE s-- no Signature of routine (e.g. prototype or parameter list)
Z scope no NONE s-- no Include the "scope:" key in scope field (use s) in tags output, scope name in xref output
a access no NONE s-- no Access (or export) of class members
e end no NONE -i- no end lines of various items
f file yes NONE --b no File-restricted scoping
i inherits no NONE s-b no Inheritance information
k NONE yes NONE s-- no Kind of tag as a single letter
l language no NONE s-- no Language of input file containing tag
m implementation no NONE s-- no Implementation information
n line no NONE -i- no Line number of tag definition
p scopeKind no NONE s-- no Kind of scope as full name
r roles no NONE s-- no Roles
s NONE yes NONE s-- no Scope of tag definition (`p' can be used for printing its kind)
t typeref yes NONE s-- no Type and name of a variable or typedef
x xpath no NONE s-- no xpath for the tag
z kind no NONE s-- no Include the "kind:" key in kind field (use k or K) in tags output, kind full name in xref output
- properties no AutoIt s-- no properties (static, volatile, ...)
- properties no C s-- no properties (static, inline, mutable,...)
- captures no C++ s-- no lambda capture list
- name yes C++ s-- no aliased names
- properties no C++ s-- no properties (static, inline, mutable,...)
- template no C++ s-- no template parameters
- properties no CUDA s-- no properties (static, inline, mutable,...)
- access yes Elixir s-- no access
- howImported no Go s-- no how the package is imported ("inline" for `.' or "init" for `_')
- package yes Go s-- no the real package specified by the package name
- packageName yes Go s-- no the name for referring the package
- assignment yes LdScript s-- no how a value is assigned to the symbol
- sectionMarker no Markdown s-- no character used for declaring section(#, ##, =, or -)
- version no Maven2 s-- no version of artifact
- category yes ObjectiveC s-- no category attached to the class
- protocols yes ObjectiveC s-- no protocols that the class (or category) confirms to
- home yes Passwd s-- no home directory
- shell yes Passwd s-- no login shell
- decorators no Python s-- no decorators on functions and classes
- sectionMarker no ReStructuredText s-- no character used for declaring section
- uri yes XML s-- no uri associated with name prefix
[jet@living]~/var/ctags%
Ideally, field related code and writer related code should be separated. I'm looking for a better way to satisfy the requirement.
@masatake Concerning my initial question, I believe I have made some progress but another question also comes up: is there any definite way I can use to see if a tag has children or not, e.g., each variable doesn't have any children.
is there any definite way I can use to see if a tag has children or not, e.g., each variable doesn't have any children.
There is no way. You can know whether an entry in a file has children or not after reading all entries in the file.
@masatake Is that possible to add a new field scopePattern
? In https://github.com/liuchengxu/vista.vim/pull/58, the nested display works well mostly, but still fails to handle some special cases, e.g., implement a struct multiple times in Rust.
The source file is https://github.com/paritytech/substrate/blob/3ec6247e44/srml/staking/src/lib.rs
$ ctags --format=2 --excmd=pattern --fields=nksSaf --extras= --file-scope=yes --sort=no --append=no --language-force=rust --rust-kinds=cPstvfgieMnm --output-format=json -f- lib.rs
The nested display generated at the moment:
As you can see, Module ...1040
actually should only have one child on_session_change
, but currently I have no way or don't know how to filter it exactly.
However, if the original tag info of on_session_change
contains the pattern of scope, then I believe I can find it exactly.
I don't know well about vim. So show the Rust input, the current tags output, and the tags that you expect.
Ok.
$ cd /tmp
$ curl -O https://raw.githubusercontent.com/paritytech/substrate/3ec6247e44ba8656007fbf75295d6d373b955846/srml/staking/src/lib.rs
$ ctags --format=2 --excmd=pattern --fields=nksSaf --extras= --file-scope=yes --sort=no --append=no --language-force=rust --rust-kinds=cPstvfgieMnm --output-format=json -f- lib.rs
There are several implementation for Module
:
the first one is from https://github.com/paritytech/substrate/blob/3ec6247e44/srml/staking/src/lib.rs#L757, its pattern is
{"_type": "tag", "name": "Module", "path": "lib.rs", "pattern": "/^impl<T: Trait> Module<T> {$/", "line": 757, "kind": "implementation"}
then next three implementations for Module
from https://github.com/paritytech/substrate/blob/3ec6247e44/srml/staking/src/lib.rs#L1040-L1065
{"_type": "tag", "name": "Module", "path": "lib.rs", "pattern": "/^impl<T: Trait> OnSessionChange<T::Moment> for Module<T> {$/", "line": 1040, "kind": "implementation"}
{"_type": "tag", "name": "Module", "path": "lib.rs", "pattern": "/^impl<T: Trait> OnFreeBalanceZero<T::AccountId> for Module<T> {$/", "line": 1046, "kind": "implementation"}
{"_type": "tag", "name": "Module", "path": "lib.rs", "pattern": "/^impl<T: Trait> consensus::OnOfflineReport<Vec<u32>> for Module<T> {$/", "line": 1058, "kind": "implementation"}
These implementations have the same name and kind, but different pattern. Without a new field like scopePattern
, I don't know how to differentiate them.
For instance, if I know the pattern of scope, I can precisely tell on_session_change
is a child of "/^impl<T: Trait> OnSessionChange<T::Moment> for Module<T> {$/"
instead of "pattern": "/^impl<T: Trait> Module<T> {$/"
.
{"_type": "tag", "name": "on_session_change", "path": "lib.rs", "pattern": "/^\tfn on_session_change(elapsed: T::Moment, should_reward: bool) {$/", "line": 1041, "signature": "(elapsed: T::Moment, should_reward: bool)", "kind": "method", "scope": "Module", "scopeKind": "implementation", "scopePattern": "/^impl
Besides the scopePattern
indicating the pattern of some tag's scope, I think scopeLine
to tell the line of its scope could be viable too. Anyway, just need a way to distinguish these scopes with same name and kind but different pattern.
I feel you are going in the wrong direction. Introduce scopePattern doesn't help you.
What we have to think about is Rust and how ctags models language objects in Rust. See https://github.com/universal-ctags/ctags/pull/1603.
I don't know Rust. Let's allow me to use a simplified input. I like tags output format than json format. So I will use tags output format.
foo.rst
1: impl<T: Trait> Module<T> {...
2: }
3: impl<T: Trait> X<T::Moment> for Module<T> {
4: fn Y(elapsed: T::Moment, should_reward: bool) {...
5: }
6: }
The current ctags implementation emits two tag entries for Module
. One is in line 1. Another is in line 3.
Both have the same kind c/class
.
Your problem is you cannot resolve Y's parent because Y's parent is just Module
.
If you have no time, the line number may help you. The line number for Y is 4. You can use the following heuristic; if there are more than one candidates as scope parents, choose one defined in the line nearest
to the child.
If you have time and want to fix this issue completely, please, read more.
The current rust parser captures Module in line 3. This is what we should fix.
Instead of capturing Module, X, X
So we can give X as a scope for Y. As a result, your issue is fixed.
The biggest issue is that what kind of name the rust parser should use for the tag taken from line 3?
I discussed this topic with @ithinuel. However, we cannot find an acceptable one.
https://github.com/universal-ctags/ctags/pull/1603 https://github.com/universal-ctags/ctags/pull/1604 https://github.com/universal-ctags/ctags/pull/1601
At that time, we discussed only how the name should be. Now I think I have to think about kind for the name, too. If we can only 'c/class' for the name, we have to rather make the name for capturing longer. However, if we can use(assign) different kind, we can use a simplified name.
impl<T: Trait> X<T::Moment> for Module<T> {...
How do you call this? How do you answer a question "what is X?" ? For my limited knowledge about Rust, X looks like a method.
If you are not interested in Rust itself, use the line: field. It will be the simplest solution.
impl<T: Trait> X<T::Moment> for Module<T> {...
fn Y...
Here X is a trait implementation attached to Module. X and Module, which is the parent of Y? Maybe both.
How about introducing trait
field to the method kind?
Module trait:X
Y scope:class:Module trait:X
or
Y scope:class:Module <T> trait:X<T::Moment>
Thanks for your time, seemingly not easy to find a quick solution. I'm actually not a master of Rust/C language, sorry for can't help more.
The approach I'm using for building the nested structure is to gather all potential root items first, and then find all it's descendants, so even using the heuristic way you mentioned to find the exact parent, I have to redesign the current code, and that will result in some more computation obviously. Anyhow, I'll see what I can do. Thanks again!
Missing your last comment :(.
How about introducing trait field to the method kind?
I think that's doable. IMHO the relation is that Y
is included in the trait X<T::Moment>
, and then X<T::Moment>
is included in Module<T>
. So the scope of Y
is trait:X<T::Moment> class:Module <T>
, or class:Module<T> trait:X<T::Moment>
.
( Thank you for contacting us.
If you are reporting an issue with the parsing output, please fill the following template. As your custom CTags configuration can affect results, please always use
--options=NONE
as the first option when runningctags
.Otherwise, delete the template and write your issue from scratch. Examples may help developers understanding your issue better.
Use GitHub web interface and markdown notation. Using mail results broken text rendering that makes the developers go crazy. )
The name of the parser: any one
The command line you used to run ctags:
The content of input file
test.py
:The tags output you:
My question is how to generate a tree like structure from the output above? Ref to (https://github.com/liuchengxu/vista.vim/issues/30) I did try to search for some document about how the scoped tags are related, e.g., how to get all the members/methods of a class, but no luck. I know the tagbar code could a reference, but I also would like to hear some instructions from the official. Thanks!
The version of ctags:
How do you get ctags binary: